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EXECUTIVE  SUMMARY 

A.  OBJECTIVES 

The  objective  of  this  study  was  to  determine  the  feasibility 

of  using  machine  vision  technology  to  effectively  and  reliably 

identify  fires  and  to  discriminate  them  from  nonfire  (false  alarm) 

sources.  The  study  was  directed  to  applying  machine  vision 

technology  to  a  number  of  assumed  Air  Force  goals  in  fire 

protection,  including:  (1)  Increased  reliability  of  fire 

detection;  (2)  Elimination  of  false  alarm  problems;  (3)  Provision 

for  faster  and  more  specific  information  on  fire  size,  location, 

type,  and  threat;  (4)  Provision  for  wider  applications  (such  as 

security  and  surveillance) ;  (5)  Provision  for  higher  performance 

capabilities  in  detecting  smaller  fires  at  greater  distances  in 

very  short  times;  and  (6)  Adaptability  to  and  compatibility  with 

current  installed  fire  protection  systems.  The  research  and 

development  performed  in  this  Phase  I  study  confirmed  that  all 

these  goals  are  attainable  with  the  Machine  Vision  Fire  Detector 

© 

System  (MVFDS)  concept,  and  that  such  a  product  can  be  readily 
developed  with  current  technology  and  off-the-shelf  hardware  at  no 
technical  risk. 

B .  BACKGROUND 

Some  problems  in  fire  detection  and  suppression  are  directly 
related  to  how  fires  are  detected.  Almost  all  of  today's  detectors 
rely  upon  the  emissions  of  ultraviolet  (UV)  and/or  infrared  (IR) 
radiations  during  combustion.  All  fires  emit  radiation  in  these 
bands.  Unfortunately,  so  do  many  other  objects  and  phenomena,  thus 
possibly  "fooling"  these  detectors  into  alarming  or  releasing 
suppressant  when  no  fire  exists.  Conventional  detection  methods 
are  "indirect"  in  that  they  do  not  know  the  nature  of  the  source, 
where  it  is  located,  its  size,  its  distance,  or  its  threat  (in 
terms  of  growth/ intensity  and  association  with  another  item) .  The 
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technique  developed  in  this  study  used  direct  methods  to 
discriminate  fire  from  nonfire  sour c'-s,  and  to  determine  the  fire's 
location,  distance,  size,  growth  and  growth  rate,  and  other 
features  that  may  be  important  in  protecting  mission  essential 
assets . 

C.  REQUIREMENTS 

The  requirements  for  most  AF  applications  involving  aircraft 
facilities  and  ground-based  systems  are  to  detect  medium  sized 
fires  of  12-100  ft2  at  about  100-foot  distance  within  b  or  so 
seconds  (after  fire  has  reached  specified  size).  In  some  instances 
these  requirements  may  not  be  stringent  enough  and  call  for 
detection  of  much  smaller  fires  at  greater  distances  in  a  time  as 
fast  as  l  second  or  so  from  fire  start.  One  possible  reason  for 
more  stringent  requirements  is  due  to  the  rapid  (10-12  ft/sec) 
spread  of  flame  across  a  JP-4  fuel  spill.  Within  only  1  second  of 
ignition,  such  a  fire  could  reach  hundreds  of  square  feet  in  area 
and  raise  temperature  at  wj.ng  height  to  1800°F  in  a  few  seebnds, 
possibly  resulting  in  aircraft  damage.  Setting  conventional 
detectors  to  higher  sensitivities  to  rapidly  detect  small  events 
also  increases  their  susceptibility  to  false  alarming  from  low 
intensity,  spurious  UV  and  IR  radiations. 

This  study  demonstrated  that  state-of-the-art  machine  vision 
technology,  combined  with  pattern  recognition,  artificial 
intelligence,  and  computer  image  processing  techniques,  provides 
the  capability  to  identify  very  small  fires  at  large  distances  in 
1  second  or  less,  and  determine  their  size,  growth,  distance,  and 
other  features. 

D .  STUDY  APPROACH 

The  approach  was  to  use  real  fire  data,  such  as  JP-4  fires,  as 
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input  to  develop  identification  and  classification  algorithms. 
During  fire  tests,  various  types  of  objects  considered  as  possible 
false  alarm  sources  were  included  in  the  scene  in  attempts  to 
"fool/confuse"  the  algorithms.  The  color  video  data  was  then 
digitized,  frame-to-frame,  and  processed  for  various  fire- 
associated  features. 

A  number  of  fire-specific  properties  were  found  that  could  be 
used  to  discriminate  fire  from  other  light  sources.  These 
properties  were  translated  into  algorithms  that  were  then  tested 
against  real  fire  data  and  false-alarm  sources  (nonfire  light 
emitters  and  reflectors) .  A  detection,  discrimination,  and 
measurement  strategy  was  then  developed. 

E.  FIRE  DETECTION  APPROACH 


The  process  which  was  selected  included  the  use  of  sensitive 
UV  and  IR  discrete  detectors  to  signal  the  presence  of  UV  and  IR 
radiation.  Prior  to  such  signals  the  MVFDS  CCD  camera  operates  in 
a  standby  mode  of  capturing  a  scene  every  few  seconds  or  so.  Each 
scene  is  digitized  and  stored  into  memory,  replacing  the  previous 
"base"  reference  frame.  Once  UV  and  IR  radiation  is  detected  in 
the  scene,  the  MVFDS  enters  "alert  mode",  whereby  the  frame  capture 
rate  increases  to  say  1  frame  every  100  milliseconds.  Immediately 
upon  entering  this  mode,  MVFDS  locates  bright  regions  that  are  not 
background  regions  and  computes  the  scene  position  corresponding  to 
these  regions.  After  identifying  these  active  regions,  MVFDS 
processes  incoming  frames  at  0.1  second  intervals  (or  some  other 
preselected  frequency)  and  for  each  new  frame  computes  the 
following  properties  for  each  active  region:  size,  growth  rate, 
stationarity ,  mean  spectral  content,  spatial  variation,  and 
temporal  variation. 

For  any  of  the  active  regions  MVFDS  can  take  any  of  these 
actions:  (1)  discontinue  tracking,  (2)  label  as  fire,  (3)  alarm 
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and/or  dump  suppressant  when  the  size  reaches  the  preset  value 
requiring  dump.  A  region  is  labeled  as  fire  if  it  exhibits 
appropriate  stationarity ,  spectral  signature,  spatial  variation, 
and  flicker.  If  it  does  not,  tracking  is  discontinued.  This 
process  was  successfully  demonstrated  at  the  conclusion  of  the 
program,  thus  proving  the  feasibility  of  the  concept  as  well  as  the 
hardware . 

F.  CONCLUSIONS 

The  MVFDS  concept  is  feasible  and  can  be  cost-effectively 
implemented  with  current  hardware.  The  concept  can  provide  a  very 
large  advance  in  fire  detection  and  protection  technology.  It  was 
recommended  that  (1)  the  Air  Force  proceed  with  diligence  to 
develop  the  MVFDS,  (2)  give  consideration  to  its  initial 
integration  into  existing  installed  fire  protection  systems,  and 
(3)  evaluate  it  for  other  applications  that  may  be  associated  with 
survivability  of  mission  essential  systems  and  operations, 
including  operational  aircraft  shelters  such  as  the  Hardened 
Aircraft  Shelters. 
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SECTION  I 


INTRODUCTION 


A.  OBJECTIVES 


The  objectives  of  this  research  were  twofold:  (1)  to  verify 
the  feasibility  of  using  machine  vision  technology  to  identify 
fires  and  discriminate  them  from  nonfire  (false  alarm)  sources;  and 
(2)  to  develop  machine  vision  concepts  that  provide  important  real 
time  information  on  fire  event  size  and  location/position.  This 
provides  for  the  first  time  a  direct  technique  (as  opposed  to 
indirect  techniques  by  current  detectors)  to  determine  actual  fire 
size  and  location,  which  can  be  used  as  the  basis  to  automatically 
release  f ire-extinguishant  agent  in  selected  "zones"  (if  total 
flooding  is  not  required)  for  preselected  sizes  of  fires.  Current 
detectors  do  not  know  where  the  fire  is  located  or  whether  it  is  a 
small  fire  near  the  detector  or  a  large  fire  some  distance  away. 

A  major  factor  underlying  the  objectives  of  this  research  is 
the  need  to  solve  the  problems  caused  by  false  alarms  due  to 
electromagnetic  emissions  from  various  types  of  nonfire  sources. 

It  was,  therefore,  a  major  consideration  in  this  study  to  develop 

.  .  .  .  © 

a  Machine  Vision  Fire  Detection  System  (MVFDS)  that  provided 

immunity  to  nonfire,  false  alarm  sources. 

In  addition  to  satisfying  the  above  objectives,  it  was  also 

deemed  necessary  to  prove  that  the  technology  exists  and  presents 

© 

minimum  risk  to  further  development  of  the  MVFDS  product. 
Demonstrations  were  provided  of  the  MVFDS  concept  and  of  candidate 
hardware  configurations,  using  currently  available  state  of  the  art 
components . 

The  third  and  final  objective  was  to  prove  the  feasibility  of 
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successfully  completing  a  Phase  II/III  effort,  and  producing  a 
product  that  would  satisfy  the  goals  of  the  USAF,  herein  assumed  to 
be : 

1.  Increase  reliability  of  fire  detection 

2.  Eliminate  false  alarm  problems 

3.  Provide  more  specific  information  on 
fire  type,  size,  location,  and  nature 

of  event,  thus  allowing  for  "intelligent" 
detector  decisions 

4.  Wider  applications 

5.  Higher  performance  specifications 

6.  Adaptable  to  and  compatible  with  current 
fire  protection  system  installations 

B .  BACKGROUND 

The  problem  of  fire  detector  false  alarms  and  accidental 
releases  of  suppressant  is  well-known,  but  certainly  not  acceptable 
(Reference  1)  .  The  nature  and  complexity  of  operations  within 
military  facilities  pose  many  potential  fire  threats  and  expose 
fire  protection  systems  to  a  large  variety  of  ultraviolet  (UV) , 
infrared  (IR) ,  visible,  and  X-ray  radiations.  High  power,  low-  and 
high-frequency  electromagnetic  radiation  from  communications  and 
aircraft  navigation  systems  are  also  present,  as  well  as  acoustic 
emissions  from  such  sources  as  aircraft  engines.  Fire  detectors 
are  expected  to  detect  fires  in  the  presence  of  such  radiation 
emissions  and,  at  the  same  time,  to  be  immune  to  these  emissions. 
This  expectation  has  not  yet  been  fulfilled. 


Many  types  of  fire  detectors  can  measure  direct  properties  and 
indirect  effects  of  fire.  This  includes  measurement  of  one  or  more 
of  the  following  physical  properties  or  phenomena:  combustion 
aerosols;  gases;  smoke/particulate  matter;  flames;  temperature; 
temperature  rate-of-rise;  and  emissions  of  UV,  IR,  and/or  visible 
radiations.  Specific  detection  technigues  vary  widely  and  include 
the  following: 

1.  Optical  flame  detectors  measure  selective  electromagnetic 
emissions  from  the  combustion  process  in  UV,  IR,  and  visible 
wavelengths.  These  are  the  most  common  detectors  used  today.  They 
can  be  fooled  by  objects/events/phenomena  that  also  emit  radiations 
in  these  wavelength  regions,  either  selectively  or  broad  band, 
and/or  reflect  radiations  in  wavelength  bands  that  emanate  from 
other  sources. 

2.  Smoke  detectors  operate  by  (a)  detecting  changes  in  light 
intensity  of  an  emitting  source  over  a  defined  path  length,  caused 
by  the  presence  of  light-scattering  particulates,  or  (b)  ionization 
changes  within  a  chamber  due  to  the  influx  of  particulates.  These 
detectors  can  be  set  off  by  the  presence  of  aerosols,  dust,  fumes, 
fog,  insects  and  obscuration  in  a  light  path.  Also,  slow-smoke 
must  reach  the  detector. 

3.  Thermal  detectors  measure  heat  from  fires  by  either  (a) 
a  solid  state  infrared  detector  or  other  device  that  is  set  to 
respond  when  a  certain  threshold  temperature  is  reached,  or  (b)  a 
rate-of-rise  of  temperature  detector  that  operates  over  a  preset 
range  of  temperature  increase  within  a  preset  time  interval.  These 
detectors  are  also  sensitive  to  hot  effluents  of  aircraft  engines 
and  aircraft  ground  equipments  (AGE) .  Instances  have  been  reported 
when  such  detectors  were  activated  by  aircraft  engine  starts  and 
aircraft  taxiing  past  an  open  hangar  door. 

Optical  flame  and  thermal  detectors  are  most  often  used  for 
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Air  Force  applications,  especially  within  aircraft  shelters  and 
aircraft  support  facilities.  However,  only  optical  flame  detectors 
apply  to  fire  threats  that  require  rapid  detection  and  suppressant 
response.  These  detectors  must  be  able  to  detect  fires  of  a  few 
square  feet  (e.g.  16  ft2)  at  some  distance  (e.g.  100  feet) ,  within 
3-5  seconds.  As  the  requirements  increase  for  detection  of  smaller 
fires  at  larger  distances  in  shorter  time  periods,  the 
susceptibility  of  the  detector  to  respond  to  "false  alarm"  sources 
also  increases.  This  undesirable  feature  of  UV  and  IR  detectors 
jeopardizes  fire  protection  when  the  fire  threat  to  which  the 
system  is  designed  dictates  a  very  rapid  and  reliable  response 
time,  say  2  or  less  seconds. 

Efforts  have  been  made  to  overcome  this  problem  of  increased 
susceptibility  to  false  alarms  versus  increased  sensitivity  to  fire 
detection  by  adding  requirements  in  the  detection  logic  such  as 
ratioing  between  UV  and  IR  intensities  and  time  gating  of  detector 
response.  These  added  features  have  provided  some  improvement  in 
false  alarm  susceptibility,  but  have  not  solved  the  problem,  as  is 
evident  by  the  continuing  frequency  of  occurrence  of  such  events 
(Reference  1) .  Other  attempts  have  included  the  addition  of  more 
spectral-specific  wavelength  measurements,  whereby  the  fire 
detector  operates  more  like  a  spectrometer. 

Other  features  of  current  fire  detectors/flame  detectors  that 
could  also  be  improved  upon.  An  important  performance  capability 
of  a  fire  detector  is  to  determine  when  a  fire  event  has  reached  a 
certain  geometrical  size.  Present  AF  requirements  (Reference  2) 
state  that  a  flame  detector  must  be  able  to  identify  a  hydrocarbon 
fire  of  size  100  ft2  (10  foot  by  10  foot  square  pan)  at  a  distance 
of  150  feet  in  5  seconds  or  less.  A  fire  of  this  size  located 
under  the  wing  of  a  fighter  aircraft  could  cause  damage  in  less 
than  5  seconds.  Such  fire  threats  may  also  have  serious  effects  on 
aircraft  structures,  including  composite  materials  as  well  as 


4 


aluminum.  Accurate  and  timely  determination  of  the  size  (and 
location,  if  possible)  of  a  fire  is  very  important  to  the 
survivability  of  certain  weapon  systems  and  mission  critical 
assets . 

The  method  employed  in  current  UV/IR  detectors  is  an  indirect 
technique  that  is  "intensity"-based.  Usually,  a  pan  fire  of  the 
size  specified  in  the  performance  requirements  of  the  purchase 
order/contract  is  set  at  the  specified  distance  from  the  detector. 
The  detector  electronics  is  then  adjusted  until  a  "fire"  signal  is 
registered.  This  process  of  setting  the  fire  threshold  level  can 
also  be  carried  out  in  the  manufacturing  facility.  The  problem 
with  this  indirect  technique  is  that  the  intensities  of  the  fire- 
emitted  UV  and  IR  radiations  are  inversely  proportional  to  the 
square  of  the  distance  of  the  emitting  source.  This  means  that 
such  a  detector  cannot  accurately  determine  the  size  of  a  fire 
because  it  does  not  know  its  distance,  yet  alone  its  dimensions. 
A  small  fire  only  a  few  feet  from  a  detector  looks  the  same  to  the 
detector  as  a  large  fire  at  much  greater  distance. 

It  is  impossible  then,  with  current  UV/IR  detectors,  to 
monitor  the  physical  growth  of  a  fire  and  to  activate  alarms  of 
different  status  depending  upon  fire  size  and,  perhaps,  location. 
For  example,  it  may  be  desirable  in  some  instances  to  know  the 
presence  of  a  fire  in  a  facility  when  the  fire  is  only  one  second 
old  and  very  small,  but  growing  rapidly  (a  JP-4  fire  spreads  at  a 
flame  front  rate  of  10-12  feet  per  second)  .  An  alarm  could  be 
activated  when  the  fire  is  very  small  to  allow  for  the  use  of  hand¬ 
held  extinguishers,  if  available.  If  the  fire  continues  to  grow, 
as  monitored  by  the  detector,  a  full  dump  of  suppressant  could  be 
programmed  to  occur  when  (if)  the  fire  reaches  a  threatening  size. 
Other  examples  include  knowing  the  association  of  the  fire  event  in 
determining  how  to  suppress  the  fire.  Jet  Fuel  Starter  (JFS) 
fires/nacelle  fires  usually  do  not  require  a  major  suppressant  dump 
because  they  can  be  extinguished  by  the  ground  crew  with  small 
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extinguishers.  However,  if  the  fire  gets  out  of  hand  and  spreads 
to  a  three-dimensional  fire  involving  the  floor,  and  the  fire 
continues  to  grow,  it  would  be  helpful  to  have  a  "smart"  detector 
that  "knows"  when  the  fire  has  reached  a  threatening  status,  and 
then  activates  the  suppressant  system.  This  capability  appears  to 
be  possible  with  the  MVFDS . 

Regardless  of  the  technique  or  concept  employed  to  date,  no 
previously  employed  or  disclosed  fire  detector  device  or  system 
concept  has  been  able  to  provide  immunity  against  false  alarms  and 
false  activations  of  suppressant,  or  provide  the  capabilities 
referred  to  above.  The  detector  concept  discussed  herein  offers 
for  the  first  time  a  capability  to  provide  immunity/discrimination 
and,  at  the  same  time,  increased  sensitivity  to  detect  fires  of 
smaller  sizes,  at  greater  distances,  and  in  much  shorter  time  than 
previously  available.  It  also  provides  the  capability  to  add 
discrimination  of  any  unforseen  new  false  alarm  source. 

The  "ideal"  fire  detector  could  be  described  as  a  human  being 
with  full  f ield-of-view  of  the  area  to  be  protected,  who  never  gets 
tired  and  could  react  with  a  decision  and  manual  response  within  a 
"blink-of-an-eye"  (0.1  seconds).  The  machine  vision  technology 
discussed  in  this  document  functions  like  a  human  being.  By 
experience,  we  "instantly"  know  when  we  see  a  fire  that  it  is  a 
fire  of  certain  size,  located  at  some  estimated  distance,  and 
possibly  associated  with  some  object.  We  recognize  fire  by  its 
brightness,  color,  color  variations,  time-intensity  changes,  shape, 
flame  edge  (tongues)  flicker,  stationarity ,  and  growth.  We  also 
know  almost  instantly  its  location  and  that  it  is  growing  in 
intensity  and  size.  If  one  had  an  electrical  switch  that  could  be 
activated  on  first  seeing  the  fire  to  sound  an  alarm,  and  another 
switch  that  could  release  the  suppressant  when  we  feel  that  the 
fire  size  has  reached  threatening  proportions,  this  would  be  a 
model  of  the  machine  vision  technology  fire  detector,  but  would  be 
much  slower,  less  accurate,  and  much  more  costly. 
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c. 


POTENTIAL  COMMERCIAL  BENEFITS  OF  MVFDS 


Technology  now  exists  to  produce  "smart"  detectors  that  can 
monitor  a  variety  of  physical  phenomena,  including  their  spatial, 
temporal,  and  spectral  characteristics.  Robotic  systems  and 
intelligence  gathering  space  platforms  such  as  Landsat  are  examples 
of  the  use  of  sophisticated  electronic  logic  in  providing 
specificity  of  the  observed  obj ect /phenomena .  Machine  vision 
technology  utilizes  one  or  more  cameras  to  obtain  image  information 
such  as  brightness,  depth,  color,  shape,  texture  and  other 
characteristics.  An  electronic  interface  between  the  optics  and  a 
computer  or  specialized  processor  is  provided  to  process  images  and 
their  information  content.  Artificial  intelligence,  pattern 
recognition,  and  image  processing  techniques  are  used  to  process 
the  incoming  images/ information.  Special  purpose  mathematical 
algorithms  are  derived  to  deduce  specific  information  needed  to 
make  certain  decisions.  The  hardware  includes  existing  camera 
devices,  simple  optics,  and  personal  computer  (PC)  components  and 
microprocessors.  All  the  necessary  hardware  for  a  machine  vision 
fire  detector  exists.  The  artificial  intelligence,  pattern 
recognition,  and  image  processing  technologies  also  exist  and  have 
been  proven  in  many  applications  more  complex  than  fire 
detect ion/discriminat ion. 

The  advent  of  the  MVFDS  will  provide  a  new  market  of  interest 
to  two  sectors  of  high-tech  industry.  Both  fire  detector 
manufacturers  and  optical  camera  manufacturers  are  viable 
candidates  to  produce  an  MVFDS  product.  Because  the  applications 
of  the  MVFDS  design  and  concept  cover  a  broad  range,  there  are  many 
potential  customer  outlets  for  its  use.  Intrusion  detection, 
motion  detection,  damage  assessment,  identification/presence  of 
persons  or  objects,  status  of  environments/facilities  (e.g.,  open 
or  closed  doors  with  aircraft  present),  fires  associated  with 
various  objects  in  various  locations,  presence  of  unwanted  objects 
such  as  UV  (e.g.,  cracked  light  lens)  and  IR  (hot  bodies/fire 
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ignition  threats)  sources,  and  discrimination  between  hydrocarbon 
and  hypergolic  fuel  fires,  are  examples  of  how  the  machine  vision 
fire  detector  could  be  used  in  multiple  or  special-purpose 
applications . 

The  present  market  for  optical  fire  detectors  consists 
primarily  of  the  defense/aerospace  complex  and  high-tech  industry. 
Many  old  technology  fire/flame  detectors  previously  installed  in 
aircraft  shelters/hangars  have  been  replaced  with  more  current  dual 
UV/IR  detectors  that  are  less  likely  to  false  alarm  (although 
certainly  not  immune  to  all  false  alarm  sources)  but  are  still 
greatly  limited  in  their  information  output.  Also,  many  fire 
protection/suppression  systems  are  being  updated  and  new  controller 
panels  and  extinguishant  plumbing  are  being  installed.  Although 
newer,  these  systems  still  do  not  provide  the  reliability  or 
functionality  that  could  be  provided  by  the  MVFDS. 

The  MVFDS  would  be  a  substantial  improvement  over  present 
conventional  detectors  and  could  be  an  obvious  replacement  of  or 
addition  to  existing  installed  systems,  especially  in  more 
demanding  complex  applications.  The  major  factor  influencing 
initial  purchase  decisions  will  be  the  degree  of  reliability  and 
performance  desired  vs.  the  fire  threat.  The  availability  of  a 
more  capable  and  reliable  detector  will  also  have  some  impact  on 
initial  purchase  for  new  installations,  especially  if  the  price  is 
similar.  Fires  as  small  as  one  or  so  square  feet  at  large 
distances  could  be  detected  and  action  taken  automatically  when  the 
fire  reached  the  specified  size,  not  seconds  after  it  reached  the 
specified  size  and  continued  to  grow.  Such  coverage  would  require 
a  multitude  of  UV/IR  detectors. 

There  is  a  real  need  for  such  capabilities  as  well  as  a  need 
for  a  new  approach  to  fire  detection  and  discrimination;  one  that 
is  reliable  in  protecting  valuable  and  mission-essential  weapon 
systems  and  assets,  as  well  as  that  is  fool-proof  and  adaptable  to 
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a  variety  of  complex  applications. 

The  cost  of  camera/ video  and  computer  component  hardware  is 
decreasing  at  a  very  rapid  rate.  New,  faster,  more  compact 
computers  are  entering  the  market  on  a  regular  basis.  The  lifetime 
in  the  marketplace  for  computer  processors,  random  access  memory 
chips,  and  integrated  logic  chips  is  very  short.  Today's  design  of 
an  MVFDS  will  certainly  cost  considerably  less  2  years  later. 

One  immediate  application  of  the  new  detector  would  be  to 
integrate  it  with  existing  installed  fire  protection  systems  so  as 
to  provide  a  very  large  increase  in  the  system's  reliability  and 
immunity  to  false  dumps/alarms.  The  MVFDS  can  be  easily 
retrofitted  into  current  configurations  and  panels.  The  existing 
UV,  IR,  or  UV/IR  detectors  could  be  used  as  "ANDS"  in  the  logic 
decision  path  and/or  as  "switches"  to  alert  the  MVFDS  of  the 
presence  of  such  radiations,  thus  turning  on  the  MVFDS  alert  mode 
(discussed  in  detail  later)  . 

Representatives  of  both  the  detector  industry  and  the 
video/optical  camera  industry  have  voiced  interest  in  manufacturing 
and  marketing  the  product.  The  actual  manufacturing  process  would 
be  simple  and  involve  only  a  few  steps.  Most  of  the  components 
would  be  obtained  as  an  OEM  manufacturer. 

D.  SCOPE/ APPROACH 

The  Phase  I  study  consisted  of  seven  major  parts:  (1) 
investigation  of  the  nature  and  properties  of  fires  and  possible 
sources  of  false  alarm/detector  confusion;  (2)  investigation  of 
artificial  intelligence,  pattern  recognition,  and  image  processing 
technologies  related  to  classification  and  discrimination  of  fire 
events;  (3)  development  of  appropriate  algorithms;  (4)  collection 
of  real  fire  data  and  other  optical/visible  stimuli  data  for  use  in 
developing  and  testing  algorithms;  (5)  selection  of  preliminary 


9 


image  classification  steps  and  software;  (6)  determination  of  a 
first-generation  hardware  configuration;  and  (7)  demonstration  of 
the  approach  selected  for  the  MVFDS  operation. 

This  study  went  much  further  than  normally  expected  of  a  Phase 
I  effort.  In  fact,  considerable  effort  was  conducted  in  the  actual 
development  of  algorithms  and  hardware  design,  thus  establishing  a 
firm  technical  basis  on  which  to  continue  in  Phase  II  full  scale 
development.  To  prove  feasibility  through  the  use  of  actual 
demonstrations,  much  of  the  mathematics  had  to  be  developed  and 
software  programs  written. 

Existing  technology  was  used  throughout  the  study  to  minimize 
technical  risk.  Commercial  high-tech  cameras,  frame  grabbers,  and 
state-of-the-art  small  computer  hardware  were  used  to  obtain  and 
process  video  data.  The  fire  data  were  obtained  from  controlled 
burns  of  JP-4  and  Aviation-A  (same  spectral  signature  as  JP-8)  jet 
fuels,  thus  representing  real  fire  threats.  In  the  development  of 
certain  algorithms,  the  color  video  data  taken  of  the  Hardened 
Aircraft  Shelter  Fire  Protection  System  tests  at  Tyndall  AFB  in 
1985  were  used.  The  false-alarm  source  data  were  obtained  and 
digitized  with  the  same  camera  and  computer  equipment  as  were  the 
fire  data.  In  some  instances  various  possible  false  alarm  sources 
were  introduced  into  the  scene  along  with  fires  for  the  purpose  of 
trying  to  "fool"  or  confuse  the  classification  and  discrimination 
algorithms.  These  sources  consisted  of  very  bright  lights,  strobe 
lights,  rotating  red  dome  lights,  chopped  reflected  colored  lights, 
and  colored  pattern  reflecting  surfaces.  In  keeping  with  the  basic 
approach  stated  above,  the  first  conceptual  hardware  model  of  the 
MVFDS  was  configured  using  current  technology,  available  hardware 
items,  devices  and  components.  No  new  technology  or  unproven,  not 
readily  available  hardware  was  postulated  for  the  end  product. 

The  scope  and  results,  then,  of  this  study  extended  from  (1) 
the  basic  analysis  of  the  properties  and  nature  of  fire,  fire 
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events,  and  false  alarm  sources,  to  (2)  the  development  of 
techniques  to  classify/discriminate  such  events/ sources 
unequivocally;  (3)  the  testing  of  these  algorithms  against  real 
data;  (4)  the  development  of  a  practical,  first-generation  hardware 
unit;  and  finally,  (5)  demonstration  of  the  high  performance  level 
of  the  MVFDS  and  proof  of  the  technical  as  well  as  product 
feasibility,  and  desirability  to  continue  into  a  Phase  II/III 
effort . 
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SECTION  II 


REQUIREMENTS 


The  MVFDS  requirements  addressed  in  this  study  consisted  of 
three  operational  performance  characteristics:  fire  detection/ 
identification,  false-alarm  discrimination,  and  system 
applications;  and  one  practical  consideration:  the  producibility  of 
the  product  with  minimum  technical  risk. 

A.  FIRE  DETECTION/ IDENTIFICATION 

The  first  and  foremost  requirement  of  the  MVFDS  is  to  detect 
a  fire  of  some  minimum  size,  at  some  maximum  distance,  and  within 
some  minimum  time  after  fire  start.  The  selection  of  quantitative 
values  for  these  three  variables  is  dependent  upon  the  application 
of  the  fire  protection  system  and,  of  course,  the  fire  threat. 

In  cases  where  the  dynamics  of  the  fire  threat  requires  very 
rapid  detection,  the  detector's  sensitivity  must  be  set  high  to 
enable  fast  response  to  small  events  (growth  rate  may  be  high) .  In 
this  scenario,  the  detector  may  be  required  to  identify  a  4  ft2  (2 
foot  X  2  foot)  pan  fire  anywhere  in  the  facility  (say  at  a  maximum 
distance  of  100  feet)  in  3  or  less  seconds  after  fire  start.  If 
the  time  was  3  or  less  seconds  after  the  fire  size  reached  4  ft2, 
the  fire  could  have  grown  to  many  hundreds  of  square  feet  (flame 
fronts  of  moving  or  large  stationary  pool  JP-4  fuel  fires  spread  at 
a  rate  of  10-12  ft/sec)  before  some  action  could  occur  to  release 
suppressant.  By  this  time,  damage  could  have  occurred  to  the 
aircraft  or  mission  essential  asset.  There  is  a  need,  therefore, 
in  many  applications  to  require  fire  detectors  and  detection 
systems  to  detect  "small"  fires  at  large  distances  when  they  reach 
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a  certain  specified  size,  not  seconds  after  they  have  attained  a 
certain  size  and  are  still  in  the  process  of  growing. 

There  are  also  applications  where  the  fire  threat  does  not 
require  automatic  suppressant  release  until  the  size  is  about  16 
ftz  or  larger.  The  recent  B-2  hangar  application  specified 
detection  of  a  12  -  16  ft2  fire  anywhere  in  the  facility  (about  100 
feet  from  detector)  in  5  seconds  (Reference  3) .  In  most  standard 
AF  applications,  the  requirement  is  to  detect  a  10  foot  X  10  foot 
(100  ft2)  pan  fire  at  150  feet  distance  in  5  seconds  (Reference  2). 
Other  published  information,  however,  quotes  a  time  of  90-120 
seconds  from  fire  start  as  the  time  required  to  have  90  percent  of 
the  fire  under  control  (Reference  4).  Such  a  long  time  would,  in 
some  instances,  result  in  total  aircraft  loss. 

Considering  all  the  various  applications  and  needs  for  high 
performance,  it  was  assumed  in  this  study  that  it  would  be  highly 
desirable  for  any  next  generation  detector  to  be  able  to  detect 
very  small  fires  (e.g.  1-4  ft2)  at  distances  of  100  feet  or  so  in 
1  second  or  less,  and  still  be  immune  to  false  alarms  and  false 
activations . 

The  ability  of  a  fire  detector  to  directly  determine  the  size 
of  a  floor  fire,  and  also  its  location,  was  deemed  to  be  a  very 
valuable  additional  detector  attribute  that  could  be  effectively 
employed  in  many  complex  fire  protection  applications,  which  cannot 
be  accomplished  with  current  fire  detectors.  This  feature  was  not 
originally  included  in  the  study  objectives  but,  as  discussed 
later,  was  accomplished  as  a  direct  fallout  of  the  algorithm 
development  effort. 

B.  FALSE  ALARM  SOURCE  DISCRIMINATION 


One  of  the  major  problems  associated  with  current  and  past 
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fire  detectors  is  their  history  of  false  alarming  to  nonfire 
sources  that  emit/reflect  radiation  in  the  same  wavelength  regions 
in  which  they  operate,  resulting  in  financial  loss  (suppressant 
replacement  cost  and  clean-up  costs) ,  environmental  impacts, 
interruption  of  operations,  and  down-time  of  mission  essential 
weapon  systems  (Reference  3)  .  In  some  instances,  these  mishaps 
were  frequent  and  actions  were  taken  to  disable  the  fire  detection 
system  when  aircraft  engines  were  started  or  run  at  high  power 
levels,  certain  aircraft  ground  equipments  (AGE)  were  present,  or 
when  certain  operations  occurred  such  as  x-raying,  welding,  and 
tests  of  navigation  radars  and  communication  systems. 

Current  flame  detectors  operate  primarily  in  ultraviolet  and 
infrared  wavelength  regions,  where  hydrocarbon  fire  has  strong 
emissions  and  where  solar  background  emissions  are  minimized  by 
atmospheric  absorption.  Emissions  from  such  objects  as  hangar 
lights,  outside  utility  lights,  aircraft  and  vehicle  lights,  tools, 
aircraft  engines,  aircraft  subsystems  and  special  devices  (e.g. 
jammers),  heaters,  hot  manifolds  of  AGE  and  vehicle  items, 
photographic  equipment,  welding  torches,  matches,  lighters,  and 
many  other  items  can  possibly  influence  UV/IR  detectors,  f.ome 
manufacturers  have  added  features  such  as  ratioing  and/or  gating 
operations  to  reduce  false  alarm  susceptibility.  This  has  helped 
to  some  degree  but  the  problem  still  remains  and  detectors  continue 
to  false  alarm. 

A  very  important  requirement  then  is  to  reduce  as  much  as 
possible  the  susceptibility  of  the  MVFDS  to  false-alarm  problems, 
and  to  hopefully  make  it  "false-alarm  proof."  Again  using  the 
human  analogy,  a  considerable  amount  of  fire-specific  information 
is  available  in  the  visible  region  and,  with  visual  data  input, 
the  brain  can  discriminate  real  fire  from  other  objects/events/ 
phenomena  through  comparison  with  previously  stored  knowledge,  such 
as  brightness,  color,  shape,  spectral  and  temporal  variations, 
growth,  flicker,  edges/tongues ,  and  stationar ity .  He  can  also 
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determine  relative  size,  location,  and  association  with  some  other 
object.  Although  UV  and  IR  radiations  are  obviously  present,  the 
human  being  does  not  need  (and  cannot  use,  other  than  possibly  high 
heat)  this  input  to  determine  presence  of  fire  and  the  properties 
of  the  fire.  The  MVFDS  operates  in  the  same  manner  with  the  same 
type  of  visible  information  input  and  processing.  However,  an 
additional  step  has  been  added  to  require  the  presence  of  UV  and  IR 
as  a  "switch"  to  initiate  a  high-speed  "event  search  mode",  and  as 
an  "AND"  in  the  logic  decision  tree. 

The  required  presence  of  UV  and  IR  was  determined  to  be  a 
possible  added  reliability  feature  in  the  fire  discrimination 
process,  at  least  at  this  time  in  the  MVFDS  development.  At 
present,  there  is  no  known  compelling  reason  to  require  the 
presence  of  UV  and/or  IR  radiation  at  any  wavelength  because  no 
known  false  alarm  sources  in  the  visible  spectral  region  have  yet 
been  identified  that  can  not  be  d?  __*minated  by  the  MVFDS  from 
fire.  Because  of  the  processing  intensity  requii  ed  for  some  of  the 
pattern  classification  algorithms  being  used,  their  presence  was 
also  considered  to  be  a  method  of  "switching"  from  slow  speed  scene 
processing  (e.g.  one  new  image  frame  every  10  seconds)  to  a  fast 
event  processing  mode  (e.g.  10-30  frames/second) .  However,  the 
option  exists  to  always  operate  the  MVFDS  in  a  fast  image/ frame 
process  speed  (e.g.  10-30  frames  per  second) .  A  detailed  timing 
analysis  and  hardware  (memory,  processor  speed,  cost,  etc.)  is 
required  to  finalize  the  scene  processing  time.  This  subject  will 
receive  further  attention  during  a  Phase  II  effort. 

If  conventional  detectors  are  set  to  very  high  sensitivity 
levels  to  enable  them  to  rapidly  detect  small  fires,  they  will  also 
be  very  sensitive  to  nonfire  UV  and  IR  emissions  and  easily  fooled. 
However,  the  MVFDS' s  UV  and  IR  detectors  can  be  safely  set  at  very 
high  sensitivity  levels.  The  MVFDS  should  not  care  if  the  UV  and 
IR  it  "sees,"  from  either  its  own  detectors  or  other  already 
installed  detectors  that  it  may  use  in  the  fire  discrimination 
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process,  are  from  real  or  false  alarm  sources.  The  visible 
information  obtained  by  the  camera,  and  processed  via  special 
algorithms,  should  be  sufficient  to  either  confirm  "fire"  or 
identify  the  source  as  being  a  nonfire  source.  This  is  a 
requirement  placed  upon  the  MVFDS  in  this  study. 

As  an  MVFDS  performance  requirement  of  not  being  fooled  by  any 
UV  and/or  IR  nonfire  source,  what  then  are  the  other  (non-UV/IR) 
possible  false  alarm  sources  that  may  confuse/fool  the  MVFDS?  As 
will  be  discussed  later,  moving  lights,  chopped  reflected  color 
lights,  strobe  lights,  moving  bright  objects,  highly  reflective 
surfaces,  and  other  optical/ visible  phenomena  must  not  be  able  to 
cause  the  MVFDS  to  make  a  wrong  decision,  or  keep  it  from  making  a 
decision.  This  MVFDS  requirement  will  be  shown  later  to  be  a  major 
attribute  of  the  new  detector. 

C.  MVFDS  SYSTEM  PARAMETERS 

The  MVFDS  must  be  able  to  function  as  a  system  with 
multiple  detection/camera  units,  depending  upon  the  application. 
In  addition  to  the  basic  requirements  discussed  above,  other  system 
requirements  that  should  be  imposed  upon  the  MVFDS  consist  of 
tolerance  to  environments  such  as  vibration,  shock,  water 
immersion,  fungus,  hazardous  atmosphere/explosion,  dust,  and  EMI. 
The  UL  or  FM  approval  is  certainly  not  sufficient  verification  that 
detectors  can  withstand  military  or  other  harsh  environments. 
These  harsh  environments  have  been  the  cause  in  the  past  for  some 
false  alarms  and  releases  of  suppressant. 

Other  features  that  the  MVFDS  should  possess  include  wide- 
f ield-of-view  coverage,  and  built-in  automatic  tests  of  window 
cleanliness  and  internal  system  electronic  "health". 

The  MVFDS  should  be  able  to  use  existing  installed  UV/IR 
detectors,  if  appropriate,  and  to  be  compatible  with  already 
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installed  fire  control/communication  panels.  It  is  not  certain  at 
this  time  if  there  would  be  any  benefit  for  the  former  ability,  but 
there  is  benefit  in  the  latter.  This  does  not  present  a  problem  as 
the  MVFDS  can  be  easily  designed  to  produce  appropriate  outputs. 

The  MVFDS  should  also  be  required  to  provide  other  types  of 
information  such  as  the  association  of  a  fire  to  specific  objects 
(e.g.  aircraft  fuselage,  AGE,  etc.),  zone  in  which  the  fire  is 
located  to  facilitate  use  of  zonal  suppression  as  opposed  to  total 
flood,  and  size  of  fire  (as  could  possibly  be  represented  by 
different  stages,  colors,  or  frequencies  of  alarms)  .  These  features 
could  be  considered  during  the  Phase  II  development  effort,  if 
required. 

D.  PRODUCIBILITY /AVAILABLE  TECHNOLOGY 

In  addition  to  the  performance  requirements  above,  a  minimum 
risk  requirement  must  be  imposed.  Minimum  risk  is  an 
understandable  objective  in  any  development  process;  so  is  minimum 
cost.  During  the  Phase  I  effort  it  was  deemed  a  requirement  that 
the  hardware  configuration  must  utilize  commercially  available 
components  such  as  those  in  the  PC  marketplace.  It  was  also  a 
design  consideration  whether  it  would  be  desirable  that  the 
hardware  technology  chosen  for  the  preliminary  design  be  in  a  state 
of  rapid  change  and  therefore  provide  major  cost  savings  for  1-2 
year  old  technology  that  still  satisfied  the  product  needs.  In 
other  words,  it  is  important  to  select  hardware  for  the  initial 
product  that  will  experience  near-future  price  reductions  which  can 
then  be  passed  on  to  the  buyers. 

The  Phase  I  effort  concluded  that  the  complete  MVFDS  concept 
can  be  incorporated  into  existing  hardware  with  no  technical  risk 
and  at  an  initial  cost  similar  to  that  of  old  technology  detectors. 
It  was  also  concluded  that  both  camera  and  computer  technologies 
are  advancing  so  rapidly  that  the  initial  MVFDS  capabilities  and 
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hardware  design  could  change  significantly  in  a  short  time  and 
experience  large  cost  reduction.  In  a  system  application  such  as 
the  B-2  hangars,  where  12-13  conventional  UV/IR  detectors  are 
required,  the  MVFDS  system  may  require  fewer  units  and  electronic 
interfaces  and  cost  about  the  same  or  less  than  a  conventional 
detector  system,  thus  providing  considerably  more  capability  for 
the  same  or  less  cost. 
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SECTION  III 


APPROACH  TAKEN  TO  PROVE  FEASIBILITY 


The  Phase  I  approach  was  to  use  existing  technology  and  real 
fire  and  false  alarm  data  to  test  technical  performance. 
Mathematical  models  of  fire  events  were  developed  along  with 
specific  algorithms  to  process  the  fire  and  false  alarm  data  and  to 
test  the  software's  abilities  to  accomplish  various  functions 
related  to  discriminating  fire  characteristics.  Analyses  were  made 
of  hardware  configurations/designs  concurrently  with  the 
development  of  the  software.  Demonstrations  were  then  made  of  the 
ability  of  the  MVFDS  to  satisfy  all  the  requirements  so  edicted  in 
this  study  and  outlined  in  Section  II,  thus  proving  the  feasibility 
of  developing  and  producing  an  advanced  machine  vision  fire 
detector  system  in  a  Phase  II  effort. 

A.  FIRE  DATA 

Only  real  fire  data  were  used  in  the  development  and  testing 
of  the  MVFDS  concept.  This  consisted  of  taking  videos  of 
controlled  pan  fires  of  JP-4  and  Aviation-A  (same  spectral 
signature  as  JP-8) .  Consecutive,  as  well  as  selected  frames  in  the 
videos,  were  then  digitized  into  a  16-bit  format  using  a  TARGA-Plus 
frame  grabber  card,  thus  providing  five  bits  each  per  red  (R)  , 
green  (G)  ,  and  blue  (B)  color  planes.  The  frame  grabber  and  a 
laboratory  grade  VCR  were  used  to  grab,  process,  and  analyze  video 
frames  exhibiting  specific  fire  information.  These  digitized  data 
were  then  used  in  the  software  algorithm  development  and  algorithm 
tests. 

Fires  were  set  in  7-inch  round,  11-inch  X  16-inch  rectangular, 
15-inch  round,  and  36-inch  X  6-inch  rectangular  pans  set  in  various 
combinations  at  distances  from  the  camera  of  18  inches  to  100  feet. 
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The  camera  (discussed  in  Section  IV  F-2)  used  in  the  data 
acquisition  was  the  Cohu  6815,  which  was  selected  as  the  state  of 
the  art  camera  hardware  for  the  initial  MVFDS  design  concept.  A 
4.5  mm,  92-degree  wide-angle  manual  exposure  lens  was  used.  The 
automatic  gain  control  (AGO)  was  turned  off  for  most  test  runs. 
The  manual  lens  "f"  stop  exposure  number  and  internal  electronic 
signal  integration  time  were  set  at  various  positions  during  the 
tests  to  determine  their  effect  upon  color  resolution  as  a  function 
of  light  intensity. 

In  addition  to  the  fire  data,  data  from  possible  false  alarm 
sources  were  also  obtained  with  the  same  video  camera  and 
digitized.  Potential  false  alarm,  or  confusion  sources  were 
included  in  scenes  along  with  fires  to  attempt  to  "fool"  the 
computer  software  processing.  Bright,  high-wattage  halogen  lamps, 
rotating  red  dome  lights,  yellow-orange  Xenon  strobe  lights, 
yellow/orange/green/red/black  striped  reflecting  signs,  and 
aluminum  foils  together  with  a  variable  speed  fan  (to  simulate  a 
signal  chopping  effect)  were  used  in  various  combinations  and 
conditions.  Some  sources  were  set  at  various  positions  in  the 
scene  with  respect  to  the  camera  and  to  locations  of  fires  (e.g. 
bright  Xenon  lamp  set  10  feet  in  front  of  camera  and  7  inch  pan 
fire  set  50  feet  away  to  determine  foreground  effects  on  small  fire 
detection) .  Others  were  hand  carried  or  run  toward  the  camera  to 
simulate  rapid  moving  lights/vehicles  in  attempts  to  fool  the 
detector.  The  reflecting  color  striped  signs  were  moved  toward  and 
away  from  the  camera  as  well  as  waved  in  front  of  the  camera  to 
simulate  a  moving  fire  truck  or  other  colored  surface.  This 
subject  is  discussed  further  in  Section  IV  D. 

The  described  fire  data  were  used  to  conduct  detailed  analyses 
of  the  nature  and  properties  of  fires  as  seen  in  the  visible 
spectrum.  Much  was  learned  from  this  research,  including  some 
characteristics  that  are  specific  to  fire  and,  to  the  knowledge  of 
che  investigative  team,  not  associated  with  any  other  light  source 
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or  reflecting  surface  in  any  condition.  These  specific, 
unequivocal  fire  characteristics  were  incorporated  into  the 
software  algorithms  used  in  the  MVFDS  logic  decision  and 
discrimination  processes. 

The  maximum  fire  threat  to  be  considered  in  the  MVFDS 
application  was  a  major  spill  from  a  fuel  truck,  dropped  wing  tank, 
or  panograph  fueling  system.  For  this  purpose,  data  from  the 
Hardened  Aircraft  Shelter  (HAS)  Fire  Protection  System  test  at 
Tyndall  AFB  in  1985  were  used.  This  consisted  of  the  spill 
(manually)  of  165  gallons  of  JP-4  on  the  floor,  directly  behind  and 
moving  toward  a  simulated  fighter  aircraft  full-scale  model. 
Ignition  points  were  set  at  three  locations  using  4  inch  pans  of 
burning  alcohol.  The  resulting  JP-4  fire  was  an  excellent  example 
of  a  major  threat.  Within  only  a  few  tenths  of  a  second  after 
ignition,  the  fire  grew  under  the  aircraft  to  a  size  of  over  100 
ft2,  and  within  2  seconds,  had  reached  a  size  of  major  proportions, 
and  a  wing  height  temperature  near  1800°  F.  The  data  used  from 
this  fire  consisted  of  video  tapes  recorded  inside  the  hangar  from 
a  height  of  about  20  feet,  looking  down  at  an  angle  of  about  45 
degrees . 

The  video  camera  used  in  the  tests  evidently  did  not  have  an 
adjustable  wfw  stop,  but  used  automatic  gain  control.  The  fire 
event,  which  was  very  intense,  was  exceptionally  bright  and  much  of 
the  color  was  "washed"  out  due  to  saturation  in  all  three  color 
planes.  Some  yellow-orange  color  was  visible  in  the  edges  of  the 
flame  profiles.  These  data  were  used  explicitly  to  develop  and 
verify  algorithms  pertaining  to  identifying/discriminating  bright 
objects,  and  determining  growth  and  growth  rate.  No  spectral 
analysis  was  made  with  this  data. 

During  the  same  HAS  test  as  discussed  above,  35mm  color  still 
slides  were  taken  of  the  event  from  the  opposite  hangar  door, 
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looking  "head-on”  to  the  aircraft  nose  and  to  the  moving  spill  of 
JP-4.  These  photographs  were  digitized  and  their  color  histograms 
used  along  with  those  obtained  from  the  controlled  pan  fire  tests 
mentioned  above,  for  development  of  spectral  discrimination 
algorithms . 

B.  IMAGE/DATA  PROCESSING  AND  ALGORITHM  DEVELOPMENT 

An  algorithmic  approach  to  fire  detection  has  been  developed 
based  on  physical  models  for  image  formation.  The  physical  models 
encompass  the  properties  of  fires  and  false  alarm  sources  as  well 
as  the  properties  of  image  sensors.  From  these  models  several 
image  observables  were  identified  that  can  be  used  to  distinguish 
fires  from  other  stimuli.  A  logic  structure  was  developed  to 
combine  the  various  sources  of  information  that  can  be  recovered 
from  images  to  reliably  identify  the  presence,  size,  and  location 
of  a  fire  event. 

Distinguishing  properties  of  fires  that  can  be  estimated  from 
sequences  of  color  images  include  intensity,  size,  stationarity , 
color,  spatial  texture,  and  temporal  flicker.  Using  these 
properties,  suppressant  can  be  dumped  on  detected  fires  that 
exhibit  sufficient  size  and  growth  rate.  Algorithms  to  compute 
these  properties  were  developed  and  implemented.  These  algorithms 
were  tested  on  several  sequences  of  fire  events  and  false  alarms. 
The  results  of  these  tests  indicated  the  effectiveness  of  the 
approach  based  upon  these  algorithms  for  fire  detection. 

C.  HARDWARE  ANALYSIS 

The  videotapes  of  the  HAS  JP-4  fire  tests  were  used  in 
analyzing  requirements  for  MVFDS  data  acquisition  rates, 
algorithms,  computational  timing  and  overall  system  response 
speeds.  These  analyses  helped  to  determine  the  equipment  necessary 
to  obtain  and  process  the  data. 


The  initial  approach  was  to  use  image  processing  minicomputer 
work  stations  to  simulate  and  test  fundamental  fire  edge  detection 
and  spectral  discrimination  algorithms.  The  work  station  software 
consisted  of  a  large  library  of  image  processing  algorithms 
available  to  process  simulated  fire  data.  The  choice  and  order  of 
selection  of  several  algorithms,  particularly  the  ability  to 
perform  change  detection  for  verifying  a  growing  high  intensity 
area,  and  spectral  intensity  discrimination  for  initial  fire 
identification,  resulted  from  these  early  study  efforts. 

Additionally,  the  results  of  the  simulated  fire  data  image 
processing  and  algorithm  testing  indicated  that  low  cost  hardware 
components  such  as  the  typical  80386  advanced  technology  class  of 
personal  computer,  equipped  with  a  frame  grabber  and  a  color  video 
camera,  could  acquire  the  real  time  data  at  speeds  as  fast  as  30  or 
more  video  frames  per  second  (0.033  second  intervals)  and  readily 
perform  these  fire  detection  algorithms  at  100  millisecond 
intervals  over  the  2-3  second  elapsed  time  of  the  HAS  fire  event. 

Several  instrumentation  cameras  were  evaluated  to  determine 
the  best  choices  for  a  very  flexible  system  for  initial  test  and 
evaluation  with  a  clear  path  to  developing  a  final  production 
design.  The  Cohu  6815  CCD  color  video  camera  with  a  large  number 
of  standardized  output  signal  formats  (NTSC,  Y-C,  analog  RGB)  was 
obtained  to  use  as  the  primary  MVFDS  R&D  visual  sensor.  Several 
frame  grabber  specifications  were  obtained  from  manufacturers  to 
compare  capabilities.  The  TARGA  Plus  card  was  obtained  to  satisfy 
the  MVFDS  performance  and  processing  requirements.  Commercially 
available  image  processing  programs  and  software  support  libraries 
of  "C"  language  drivers  and  graphic  processing  routines  were  also 
obtained.  The  above  hardware  and  software  were  then  installed  in 
a  80386  PC  for  processing  selected  fire  and  false  alarm  video  data 
and  demonstration  of  MVFDS  feasibility.  In  support  of  this 
computer  capability,  a  laboratory  grade  video  recorder  and  monitor 
were  included  in  the  investigative  system  to  enable  frame  freeze 
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capture  and  analysis  of  fire  data. 

Further  live  fire  tests  were  performed  using  the  Cohu  camera 
to  capture  fire  events  on  tape.  The  aperture  and  the  internal 
electronic  integration  time  were  varied  for  fire  distances  up  to 
100  feet  from  the  camera  to  determine  optimum  exposure  settings. 
Several  frame  sequences  were  digitized  from  the  new  fire  event 
tapes  to  verify  spectral  and  spatial  characteristics. 

D.  DEMONSTRATIONS 

Two  demonstrations  were  made  during  the  study.  The  first  one 
occurred  early  in  the  project  and  exemplified  existing  software 
concepts  related  to  frame  subtraction,  growth,  and  other  features 
associated  with  pattern  recognition  and  image  processing 
technologies.  The  purpose  of  this  first  demonstration  was  to  show 
that  the  base  mathematical/ computer  processing  technology  existed 
and  could  be  easily  adapted  to  the  MVFDS . 

A  second  in-depth  and  comprehensive  demonstration  was  given  at 
the  end  of  the  program.  This  demonstration  consisted  of  six  parts: 
(1)  Presentation  of  the  objectives,  approach,  fire  discrimination 
factors,  false  alarm  source  factors  and  system  performance  factors 
used  in  the  study;  (2)  Presentation  and  demonstration  of  fire  data 
used  in  the  study;  (3)  Presentation  of  the  mathematics,  algorithms, 
and  computer  processes  used  by  the  MVFDS  for  fire  discrimination 
and  false  alarm  identification;  (4)  computer  demonstration  of  each 
and  every  step  used  by  the  MVFDS  in  its  logic  tree  decision 
process;  (5)  Presentation  of  conclusions  and  examples  of  time 
profiled  events  and  how  the  MVFDS  discriminates  fire  from  nonfire 
sources;  and  (6)  Presentation  of  hardware  design  and  configuration. 

The  above  final  demonstration  covered  in  detail  the 
mathematical  models  and  algorithms  that  were  developed  during  the 
Phase  I  effort. 
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SECTION  IV 


TECHNICAL  DISCUSSION  AND  ACCOMPLISHMENTS 


This  section  discusses  the  technical  basis  of  the  operation  of 
the  MVFDS^and  results  obtained  during  the  Phase  I  effort.  The 
system  hardware  concepts  are  also  discussed. 


A.  STATE  OF  THE  ART  OF  MACHINE  VISION  TECHNOLOGY 

Machine  vision  systems  have  been  in  use  for  military  and 
commercial  applications  for  well  over  two  decades.  Coupled  with 
image  processing  and  image  classification  algorithms  and 
software,  they  have  been  applied  to  tasks  including: 

1.  Visual  inspection  for  manufacturing 

2.  X-ray  imagery  analysis  for  detecting  tumors,  bone 
fractures  and  defects  in  soft  tissue. 

3.  Parts  sorting  and  screening  for  subsystem  assembly. 

4.  Robotic  manipulator  control. 

5.  Targeting  and  photo  interpretation  support  in 
various  DOD  applications. 

1.  Commercial  Applications 

Machine  vision  has  been  applied  successfully  to  the 
problem  of  automatically  inspecting  parts  at  various  stages  in  the 
manufacturing  process.  Visual  inspection  is  an  effective  means  of 
detecting  many  different  kinds  of  defects.  Recent  technological 
advances  in  computer  hardware  and  sensors  have  made  automatic 
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visual  inspection  systems  cost  effective  for  a  wide  range  of 
industrial  applications.  Vendors  have  started  producing  flexible 
systems  that  can  be  programmed  for  various  inspection  problems. 
Custom  systems  are  now  available  for  inspection  tasks  that  cannot 
be  performed  manually. 

In  the  past  10  years,  a  great  deal  of  effort  has  been 
devoted  to  developing  systems  to  inspect  printed  circuit  boards. 
Current  systems  are  available  to  locate  many  kinds  of  defects  such 
as  shorts,  opens,  over-etching,  and  spurious  metal  before  the 
component  insertion  and  soldering  processes.  Such  systems  are 
capable  of  high  throughput,  high  detection  accuracy,  and  low  false- 
alarm  rates. 

Since  the  seventies,  many  systems  have  been  built  to 
inspect  integrated  circuit  (IC)  photomasks  that  transfer  patterns 
for  semiconductor  lithography.  The  systems  successfully  find 
defects  such  as  registration  errors  and  dimension  variations  at 
high  speeds.  Several  commercial  systems  are  available  and  used  by 
IC  manufacturers. 

Systems  have  also  been  developed  that  inspect  integrated 
circuit  chips.  Such  systems  can  check  line  widths  and  defects  such 
as  contamination  and  voids. 

Autonomous  vehicles  can  be  guided  by  machine  vision. 
Such  applications  are  difficult  because  the  vehicle  must  plan  its 
actions,  perceive  its  usually  complex  environment,  and  adapt  to 
changes  in  the  environment  and  new  situations.  Despite  these 
difficulties,  autonomous  vehicles  have  been  developed  that  are 
capable  of  road  following  and  other  tasks. 

Machine  vision  is  useful  for  many  robotics  applications. 
These  typically  involve  recognizing  and  locating  objects  for  such 
tasks  as  automated  assembly.  Available  systems  have  successfully 
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dealt  with  domains  where  objects  are  spatially  unconstrained  and 
may  occlude  each  other. 

Many  advancements  have  been  made  in  the  processing  of 
medical  images.  The  ability  of  humans  to  interpret  these  images  is 
often  limited  by  obscuration,  distortion,  or  blurring.  Machine 
vision  techniques  are  currently  used  in  medical  image  processing 
for  enhancement,  detection,  compression,  measurement, 
visualization,  and  reporting. 

2.  Military  Applications 

The  military  machine  vision  applications  include  change 
detection  to  highlight  military  unit  redeployment  as  viewed  from 
airborne  or  spaceborne  TV  and/or  IR  imagery;  damage  assessment 
following  an  engagement  or  disturbing  phenomena;  or  development 
patterns  associated  with  military  or  civilian  build-up  (roads, 
buildings,  etc.).  Similarly,  machine  vision  systems  have  a  variety 
of  military  reconnaissance  and  surveillance  applications.  These 
are  particularly  applicable  to  site  security  surveillance  for 
intrusion  detection  and  alerting  to  incoming  threats  against  ships, 
vehicles  or  military  installations. 

In  the  late  sixties,  military  change  detection  systems 
were  developed  for  optical  platforms  that  permitted  scanning, 
digitizing,  and  storing  an  image  frame,  then  comparing  this 
reference  image  with  a  frame  of  imagery  taken  at  a  later  time.  The 
system  checked  for  registration  points  on  the  image  scenes,  then 
compared  these  references  to  generate  a  change  detected  frame.  The 
system  proved  effective  in  highlighting  changed  areas  and  was  used 
for  a  variety  of  post-mission  analysis  tasks.  Evolution  of  this 
concept,  and  the  image  processing  hardware  and  software,  led  the 
way  to  the  development  of  imagery  screening  systems  to  accommodate 
the  high  volume  of  data  acquired  by  satellite  imaging  systems.  The 
advances  made  in  optical  and  digital  image  processing  spawned  a 
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number  of  video  analysis  workstations  with  continually  augmented 
capability,  including  image  enhancement,  target  recognition,  and  a 
corollary  to  change  detection,  moving  target  indication  (MTI) . 

As  with  change  detection,  detecting  motion  on  video 
imagery  required  that  a  reference  scene  be  generated,  the 
subsequent  scene  be  registered  and  compensated  for  scene 
illumination  variation,  then  subtracted  from  the  reference.  With 
MTI,  a  change  in  the  image  is  tracked  frame  to  frame  to  detect 
object  motion  (and  velocity) .  This  technique  is  used  in  many 
optical  sensor  targeting  systems,  particularly  to  cue  an  operator 
of  a  possible  threat  or  target. 

A  related  area  of  machine  vision,  and  one  that  has 
achieved  the  most  emphasis  in  the  past  decade,  is  that  of  automated 
imagery  analysis  and  target  recognition.  Optical  imagery  is 
processed  using  feature  extraction  and  statistical  pattern 
recognition  methods  to  detect,  track  and  attempt  to  identify 
targets  in  thermal  and  video  imagery.  Numerous  systems  are 
currently  employed  in  missile  guidance  systems  and  electro-optical 
tracking  systems  using  a  variety  of  signal  processing  and  pattern 
recognition  algorithms.  Real-time  hardware  implementation  is 
realized  using  high-speed  DSP  and  pipe  line  processing  systems. 
This  trend  and  development  effort  have  enabled  the  application  of 
this  technology  to  problems  such  as  fire  detection.  Combining  the 
developed  methods  of  image  processing,  image  registration  and 
subtraction,  and  image  analysis  with  the  continuing  reduction  in 
the  cost  and  size  of  the  required  hardware,  allows  a  timely  and 
cost  efficient  solution.  Machine  vision  technology  has  been  used 
in  many  complex  problems,  much  more  severe  and  complicated  than 
fire  detection  and  discrimination.  The  technology  exists,  has  been 
proven,  and  does  not  provide  any  technical  risk  for  the  development 
and  application  of  the  MVFDS . 
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Machine  Vision  Benefits  to  Fire  Detection 


3  . 


Machine  vision  technology  can  readily  and  effectively  be 
applied  to  fire  detection.  Using  a  physical  model  for  fire  events, 
it  has  been  shown  (see  Section  IV  C)  that  sequences  of  visible 
color  images  contain  sufficient  information  to  reliably  identify 
and  locate  fire  events.  Such  image  sequences  can  be  obtained  using 
a  standard  CCD  color  video  camera  and  digitizing  hardware.  The 
relevant  information  in  these  images  can  be  extracted  using  the 
machine  vision  algorithms  derived  from  fire  models  developed  in 
this  study. 

In  many  respects,  fire  detection  is  a  simpler  application 
than  many  of  the  problems  for  which  machine  vision  has  been 
successful.  As  compared  to  most  inspection  problems  where  a  system 
must  distinguish  several  different  materials,  fire  detection 
involves  only  distinguishing  fire  events  from  false  alarm  sources. 
The  many  attributes  of  a  fire  event  such  as  intensity,  size, 
growth,  stationarity ,  color,  texture,  and  flicker  that  can  be 
derived  from  the  fire  models  herein  make  a  fire  event  relatively 
easy  to  detect  in  sequences  of  images.  The  projection  model  for 
the  imaging  system  developed  in  this  study  makes  it  straightforward 
to  compute  the  scene  location  of  identified  fires  in  three 
dimensions . 

B.  DETECTION  METHODOLOGY 


During  this  study,  a  number  of  characteristics  of  fire  were 
reviewed  for  their  uniqueness  in  discriminating  fire  from  other 
objects,  events,  and  phenomena  that  may  have  some  similar 
characteristics.  The  selected  characteristics  were  analyzed  and 
tested  with  real  fire  data  and  with  false  alarm  source  data  to 
attempt  tc  "fool"  the  software  identification  criteria.  The 
following  general  characteristics  were  included  in  the  detection/ 
discrimination/classification  algorithms  as  major  steps  in  the 
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logic  decision  tree:  (1)  intensity  level  of  "bright"  area;  (2) 
color  of  bright  area;  (3)  growth  and  growth  rate;  (4) 
continuity/stationarity;  (5)  edge  profile  variations;  (6)  certain 
spectral  characteristics  within  a  frame  and  spectral  flicker  from 
f rame-to-frame;  and  (7)  presence  of  UV/IR.  After  all  these 
conditions  are  satisfied,  the  fire  alarm  is  activated.  The  MVFDS 
continues  to  monitor  the  fire  event  until  its  size  and  other 
characteristics  reach  the  predetermined  requirements  for  size 
and/or  association  with  some  object  to  warrant  an  automatic  release 
of  suppressant. 

The  MVFDS  relies  on  standard  silicon  charge  coupled  device 
(CCD)  color  camera  outputs  in  three  color  (red,  green,  blue)  analog 
signals.  As  previously  discussed,  the  MVFDS  simulates  the 
detection  and  logic  process  of  a  human  being,  using  only  visual 
sensory  input  data  in  the  visible  region.  The  video  camera  outputs 
one  frame  at  a  time,  encompassing  the  entire  scene  as  covered  by 
the  camera's  f ield-of-view  (chosen  here  to  be  90  or  more  degrees). 
The  frame  rate  is  predetermined  by  the  amount  of  information 
required  to  be  processed  within  some  interval  of  time.  For 
purposes  herein,  the  MVFDS  operates  at  a  standby  rate  of  one  frame 
every  1-10  seconds  until  it  switches  to  a  fast  "event  mode"  where 
the  processing  rate  is  increased  to  10-30  frames  per  second. 

The  MVFDS  uses  one  or  more  (in  a  system  configuration)  cameras 
to  routinely  monitor  the  area/ volume  being  protected.  A  new 
frame/scene  is  "grabbed",  digitized,  and  stored  in  memory  every 
1-10  seconds,  replacing  the  previous  frame.  This  process  can 
continue  indefinitely  until  some  change  or  characteristic  in  the 
scene  occurs  for  which  the  MVFDS  was  preprogrammed  to  identify. 

Encompassed  within  the  MVFDS  are  small  UV  and  IR  detectors 
that  are  very  sensitive  to  wavelengths  at  200-300  nanometers  and 
4.3  micrometers,  respectively.  They  are  purposefully  set  at  very 
high  sensitivities  to  instantly  respond  to  any  UV/IR  emitting 


source.  The  presence  of  these  wavelength  emissions  during 
hydrocarbon  fire  events  is  the  basis  on  which  conventional  flame 
detectors  depend.  As  stated  earlier,  because  of  this  dependency 
and  sensitivity  they  are  also  likely  to  alarm  to  a  variety  of 
nonfire  sources,  of  which  there  are  many. 

These  detectors  are  used  by  the  MVFDS  as  a  precursor  switch  to 
increase  the  frame  processing  rate  ("alert/event  mode") .  The  MVFDS 
does  not  care  if  the  UV  and  IR  detectors  see  an  actual  fire  or  not, 
because  confirmation  is  made  by  computer  processing  of  the  camera 
data.  Immediately  upon  registering  the  presence  of  these 
radiations,  the  MVFDS  initiates  an  alarm  (if  desired)  in  the 
facility  that  a  UV/IR  source  exists  and  a  new  frame  is  grabbed, 
digitized,  processed  and  stored.  This  frame  now  becomes  the  new 
base  frame,  replacing  the  previous  frame  in  memory.  Another  frame 
is  then  obtained,  say  within  l/10th  of  a  second  or  sooner.  The 
computer  processing  performed  on  each  frame  and  between  frames  is 
described  as  follows. 

1.  The  first  frame,  grabbed  after  the  UV/IR  detectors  have 
activated  the  event  mode,  is  processed  for  bright  areas  consisting 
of  pixels  above  threshold  intensity  levels  in  preselected  fire 
associated  color  bands.  If  such  a  bright  area  is  identified,  the 
pixels  it  occupies  are  registered  in  position  coordinates  and  the 
scan  line  corresponding  to  the  base  of  the  bright  area,  on  the 
floor,  is  also  registered.  The  distance  of  the  bright  area  from 
the  camera  is  automatically  determined  via  a  stored  lookup  table 
that  references  each  scan  line  number  according  to  its  calibrated 
distance  from  the  detector  unit.  The  calibration  is  accomplished 
at  installation  with  the  use  of  markers  set  at  known  distances  on 
the  floor  which  correspond  to  the  camera  scan  line  numbers.  The 
size  of  the  bright  area  is  also  known  because  the  size  of  pixels  in 
the  area  are  automatically  known.  The  bright  area  is  edge  enhanced 
and  edge  profiled  and  this  information,  along  with  position 
information,  is  stored. 
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If  no  bright  areas  are  found  in  the  first  alert  mode 
frame,  consecutive  frames  are  grabbed  and  processed  until  one  is 
identified  or  until  the  MVFDS  decides  that  no  visible  fire  event 
exists  and,  therefore,  the  UV/IR  signals  came  from  a  false  alarm 
source.  The  elapsed  time  before  the  MVFDS  decides  that  an  event  is 
a  false  alarm  and  returns  to  standby  mode  can  be  adjusted, 
depending  on  the  nature  of  the  fire  threat. 

2.  For  example,  assume  a  bright  area  is  identified  in  the 
first  event  mode  frame  above.  One  tenth  of  a  second  later  the  next 
frame  is  grabbed,  digitized  and  processed  in  the  same  manner  as  the 
previous  frame.  This  frame,  say  F2,  is  subtracted  from  FI,  the  new 
base  frame,  and  the  size  of  the  remainder  of  the  subtracted  bright 
area  determined.  Growth,  if  any,  and  growth  rate  are  then  known 
over  the  last  one  tenth  second.  The  edge  profile  is  determined  as 
well  as  certain  spectral  features  within  the  profile  of  the  bright 
object.  Certain  pixel  continuity  features  are  compared  frame-to- 
frame.  All  this  information  is  stored  and  position  referenced. 

3.  The  next  frame,  F3 ,  is  grabbed,  subtracted  from  frame  F2 , 
and  the  bright  area(s)  processed  the  same  as  above  for  growth, 
growth  rate,  edge  frequency  flicker,  pixel  continuity  between 
frames,  and  certain  characteristics  of  the  spectral  signatures 
within  frame  F3  and  in  comparison  to  pixels  in  frame  F2  and  FI. 

4.  The  process  continues  with  frames  F4 ,  F5,  etc.,  until  the 
preset  conditions  that  uniquely  discriminate  fire  from  other 
objects/sources/phenomena  are  all  satisfied  and  the  size  of  the 
identified  fire  area  reaches  the  predetermined  specified  size  for 
the  MVFDS  to  activate  the  suppressant.  Assuming  a  fire  threat 
similar  to  the  HAS  165  gallon  JP-4  spill/fire,  the  growth  and  other 
spatial,  temporal,  and  spectral  characteristics  could  be  satisfied 
within  only  0.3-0. 4  seconds  after  fire  start,  and  the  MVFDS  would 
be  certain  that  the  event  is  a  real  fire.  This  would  be 
investigated  further  in  a  Phase  II  effort. 
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Once  fire  is  determined,  appropriate  alarms  are  activated, 
suppressant  dump,  however,  can  be  delayed  until  the  fire  size  has 
grown  to  the  specified  threshold  for  automatic  dump.  There  are 
other  options  at  this  point,  including  zonal  suppression  since  the 
location  of  the  fire  is  known.  Other  options  may  include  alarm 
only  for  some  time  to  allow  for  manual  extinguishment,  especially 
if  the  fire  or  its  location  does  not  warrant  a  total  or  partial 
dump. 


C.  IMAGE  AND  DATA  PROCESSING  METHODOLOGY 

The  algorithmic  approach  taken  to  fire  detection  is  derived 
from  physical  models  for  the  formation  of  images  of  fires  and  other 
stimuli.  These  models  incorporate  the  physical  characteristics  of 
both  events  in  the  world  and  image  sensors.  These  physical  models, 
can  be  used  to  quantify  various  properties  derived  from  color 
images  that  can  be  used  to  reliably  distinguish  fires  from  other 
events.  These  properties  can  be  computed  at  high  speed  and,  along 
with  a  decision  procedure,  form  the  basis  of  the  fire  detection 
system.  The  effectiveness  of  these  properties  for  fire 
identification  has  been  demonstrated  on  several  sequences  of  images 
of  fires  and  false  alarm  data. 

The  imaging  system  consists  of  a  color  CCD  camera  and  a  lens. 
The  lens  guarantees  that  each  visible  point  in  the  world  will 
project  to  a  unique  point  in  the  image  according  to  the  perspective 
transformation.  The  two  dimensional  spectral  irradiance  function 
recorded  in  the  image  plane  will  be  proportional  to  the  scene 
radiance  over  corresponding  patches  in  the  scene.  The  CCD  imager 
consists  of  a  two  dimensional  array  of  collection  sites  that  are 
sensitive  to  light.  We  can  model  a  measured  color  pixel  at  (x,y) 
in  the  image  by  the  triplet  (SR,  SG,  SB)  where 
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©  c 

SR(x,y)  =  JI(x,y,  X)  fR(X)  dX 
SG{x,y)  =jl(x,y,X)fG(X)dX 
SB(x,y)  =fl(x,y,  X)  fB(X)  dX 

and  where  I(x,y,X)  is  the  incident  spectral  irradiance  at  (x,y)  and 
fR(X),  fc(X)  ,  and  fB(X)  are  the  response  of  the  red,  green,  and  blue 
sensing  elements  respectively.  The  response  of  a  sensing  element 
is  equal  to  the  transmission  of  the  color  filter  (in  this  case  red, 
green,  and  blue)  times  the  quantum  efficiency  of  the  CCD.  Typical 
curves  fR(X),  fG(X)  ,  and  fB(X)  are  shown  in  Figure  1. 
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Wavelenth  (nm) 

Figure  1.  Typical  Spectral  Response  Curves 

The  spectral  radiance  of  a  fire  in  the  scene  can  be  modelled 
by  the  equation 

£(x\y^,0  =  £(A)  +  V(x',y',t,A)@ 

where  (x',y')  denotes  coordinates  in  the  scene  projecting  to  the 
image,  the  function  E(X)  describes  the  mean  spectral  radiance  of  a 
fire,  and  the  process  V(x',y',t,X)  quantifies  spatial  variation 
(x',y'),  temporal  variation  or  flicker  (t) ,  and  color  variation  (X) 
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in  the  appearance  of  a  fire.  In  terms  of  this  model,  the  measured 
pixel  values  for  a  fire  are  given  by 


©  5*(x,y,i)  =  J  E(x',y',\,t)fR{\)d\ 
Sc{x,y,t)  =  J  E(x\y',X,t)fcWd^ 
=  J  E(x',y\A,t)fB(A)d* 


Using  this  model,  the  image  of  a  fire  will  exhibit  the 
properties  of  (1)  high  intensity,  (2)  characteristic  mean  spectral 
content,  (3)  characteristic  spatial  variation,  and  (4) 
characteristic  flicker.  A  fire  event  that  must  be  suppressed  will 
exhibit  all  of  these  four  properties,  plus  a  size  and  growth  rate 
above  a  specified  threshold  and  stationarity .  Algorithms  to 
compute  these  properties  are  described  in  following  sections. 

1.  Fire  Detection/Fire  Event  Detection 

An  overview  of  the  MVFDS  logic  structure  is  illustrated 
in  Figure  2.  The  system  operates  in  a  standby  mode  before  UV/IR 
radiation  is  detected  and  shifts  to  alert  mode  after  UV/IR 
radiation  is  detected.  The  algorithms  associated  with  individual 
decisions  within  the  system  are  described  in  Sections  IV  C-2 
through  IV  C-8 

If  UV/IR  radiation  has  not  been  detected  in  the  scene, 
the  MVFDS  operates  in  standby  mode.  In  standby  mode,  the  system 
monitors  for  bright  regions  in  the  image.  Such  bright  regions 
which  have  persisted  for  a  certain  length  of  time  are  assumed  to 
not  correspond  to  fire  threats  and  will  be  called  background 


35 


STANOaRO  MOOE 

(NO  UV/lfi> 


r 


ALERT  PIOOE 

(uv/ifl) 


Find  orignt  candidate  fire 
regions  and  locations 


>'  candidate 
fire  regions 


Grao  next  frame  <0  1  second  intervals) 

Discontinue  tracking 

laoei  region  as 

Dump  suppressant 

region  if 

fire  if 

if  fire  region  nas 

1)  not  stationary 

i )  stationary 

l )  Sufficient  size 

on 

AND 

AND 

2)  wrong  color 

2)  appropriate  color 

2)  sufficient 

OR 

AND 

growth  rate 

3)  wrong  texture 

3)  appropriate  texture 

on 

4)  no  flicker 

ANO 

4)  appropriate  flicker 

>i  candidate 
fire  regions 

no  canoioate 
fire  regions 

Figure  2.  Overview  of  MVFDS  Logic  Structure 

regions.  Tracking  background  regions  in  standby  mode  will  ensure 
that  they  are  not  considered  as  potential  fires  after  UV/IR 
radiation  has  been  detected. 


Once  UV/IR  radiation  is  detected  in  the  scene,  the  MVFDS 
enters  alert  mode.  Immediately  upon  entering  this  mode,  MVFDS 
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locates  bright  regions  R*  that  are  not  background  regions  and 
computes  the  scene  position  corresponding  to  these  regions.  After 
finding  these  active  regions,  MVFDS  processes  incoming  frames  at 
0.1-second  intervals  and,  for  each  new  frame,  computes  the 
following  properties  for  each  active  region:  size,  growth  rate, 
stationarity,  mean  spectral  content,  spatial  variation,  and 
temporal  variation. 

Following  each  frame  for  any  of  the  active  regions  MVFDS 
can  take  any  of  the  actions  (1)  discontinue  tracking,  (2)  label  as 
fire,  (3)  dump  suppressant.  Tracking  is  discontinued  for  any 
region  that  fails  to  exhibit  appropriate  (1)  stationarity,  (2) 
spectral  signature,  (3)  spatial  variation,  (4)  flicker.  A  region 
is  labeled  as  fire  if  it  exhibits  appropriate  (1)  stationarity,  (2) 
spectral  signature,  (3)  spatial  variation,  and  (4)  flicker. 
Suppressant  is  activated  for  regions  that  have  been  labeled  as 
fires  and  which  exhibit  appropriate  size  and  growth  rate. 

2.  Identification  of  Bright  Areas 

Once  the  system  enters  alert  mode,  bright  regions  that 
are  not  background  regions  must  be  identified.  Such  regions  will 
be  called  initial  active  regions.  To  find  these  regions,  MVFDS 
grabs  frames  until  regions  are  found  with  an  intensity  in  the  red 
band  that  exceeds  a  predetermined  threshold  value.  For  each  of 
these  initial  active  regions,  MVFDS  constructs  a  bounding  box  and 
computes  the  base  of  the  corresponding  event  in  the  scene  as  shown 
in  Figure  3 . 

The  position  of  regions  in  the  image  can  be  localized 
according  to  the  resolution  of  the  CCD  sensor.  A  typical  CCD 
containing  512  x  512  collection  sites  is  shown  in  Figure  4. 


512  Pixels  (1 0.25  mm) 


Figure  3.  Finding  Initial  Active  Regions 


Output  Node 


Figure  4.  Typical  CCD  Sensor  Array 
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-H  2C  nm  1-4 


Each  point  P1  in  the  sensor  plane  is  the  image  of  a  point  P  in  the 
scene  that  lies  on  the  ray  from  P*  through  the  center  of  projection 
of  the  imaging  system  (Figure  5) . 


Figure  5.  Projection  Model 


The  base  of  the  event  in  the  scene  may  be  computed  by  using  a 
lookup  table  that  models  the  projection  of  the  imaging  system  and 
by  assuming  that  the  base  of  the  scene  event  is  located  on  the 
floor.  This  provides  the  location  distance. 

3.  Determination  of  Size 

The  size  of  a  region  in  the  image  is  computed  by  counting 
the  number  of  pixels  in  the  region  that  exceed  a  threshold  in  the 
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red  band.  From  this  pixel  count  and  the  inferred  distance  of  the 
event  from  the  camera,  MVFDS  computes  the  area  of  the  event  in  the 
scene.  Computing  the  scene  area  corresponding  to  an  image  region 
requires  consideration  of  the  projection  geometry  shown  in  Figure 
5.  Each  pixel  subtends  a  certain  solid  angle  n  with  respect  to  the 
center  of  projection  of  the  imaging  system.  Thus,  each  pixel 
corresponds  to  this  solid  angle  in  the  scene  as  viewed  from  the 
center  of  projection.  The  area  A  in  the  scene  imaging  to  this 
pixel  that  is  normal  to  the  viewing  direction  is  given  by 

A  =  R2n 

where  R  is  the  distance  of  the  event  in  the  scene  from  the  camera. 
Using  this  relationship,  the  scene  area  of  an  event  may  be  computed 
from  the  image  pixel  count.  Figure  6  shows  an  example  of  the  scene 
area  computation  for  two  regions  with  similar  pixel  counts  but 
different  distances. 


R,:  AREA  30  FT2 
R2:  AREA  10  FT2 


Figure  6.  Scene  Area  Computation 
4.  Determination  of  Growth  and  Growth  Rate 


Since  the  size  of  each  region  is  computed  for  each  new 


frame,  it  is  straightforward  to  compute  the  average  growth  rate  for 
each  active  region  Rs .  The  average  growth  rate  of  region  R;  up  to 
time  t  is  defined  by 


G,  = 


Size(Ri)t  —  Size(Ri)  q 
t~-  t0 


where  t0  is  the  time  at  which  the  initial  region  Rj  was  identified. 
Figure  7  gives  an  example  of  the  growth  rate  computation  for  two 
regions. 


Figure  7.  Growth  Rate  Computation 
5.  Determination  of  Stationarity 

Stationarity  measures  the  extent  to  which  an  event 
remains  in  a  single  place  in  the  scene.  Fires  which  pose  threats 
will  exhibit  strong  stationarity  since,  even  though  they  may  grow, 
the  fire  will  continue  to  occupy  the  area  where  it  started  in  the 
scene.  Many  false  alarm  sources,  such  as  moving  headlights,  do  not 
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exhibit  strong  stationarity .  For  a  region  Rj  at  time  t, 
stationarity  is  quantified  by 

S,(t)  =  fraction  of  pixels  in  initial  region  occupied  at  time  t 

A  threshold  value  of  stationarity  is  used  to  determine  if  a  region 
R,-  is  sufficiently  stationary  to  correspond  to  a  fire  event. 
Figure  8  illustrates  the  stationarity  computation  for  two  regions. 


R,:  S(t)  =  0 
R2:  S(t)  =  1 


t°  t 

Figure  8.  Stationarity  Computation 

6.  Determination  of  Spectral  Signature 

Fires  will  have  a  characteristic  mean  color  or  spectral 
signature  given  by  the  function  E(X) .  Using  the  color  imaging 
system  measurements  of  E(X)  integrated  with  the  response  function 
of  the  sensing  elements  fR,  fG,  and  fB,  can  be  obtained.  As 
described  by  the  function  V(x‘  ,y'  ,t, X)  ,  fires  will  also  have  a 
characteristic  spectral  variation. 

The  spectral  properties  of  a  fire,  as  measured  by  a  color 


image  sensor,  may  be  characterized  by  the  probability  density 
p(R,G,B)  of  fire  pixels  in  the  digitized  sensor  measurement  space. 
Since  sufficient  data  are  usually  not  available  to  accurately 
estimate  p(R,G,B)  explicitly  as  a  three-dimensional  function,  it  is 
necessary  to  exploit  the  structure  of  p(R,G,B)  to  allow 
characterization.  An  efficient  and  accurate  parametric 
representation  for  many  color  distributions  p(R,G,B)  is  the 
multivariate  normal  density  given  by 


p[A)  = 


1 

(2n)is\EA\Q*e 


where  A  =  (R, G, B)  indicates  the  sensor  measurement,  A  =  (R,G,B)  is 
the  mean  vector  of  the  distribution,  and  is  the  covariance 
matrix  of  the  A  distribution  that  describes  the  dispersion  of  the 
data  in  any  direction  in  color  space.  Contours  of  constant  density 
are  then  ellipsoids  satisfying 


(A-A)tE^(A-A)=C 


where  C  is  a  constant.  For  fire  pixel  classification,  a  threshold 
T  is  selected  to  define  the  ellipsoid  so  that  vectors  A  for  which 


( A-A)tX-'(A-A ) 


<  T 


are  within  the  ellipsoid  and  considered  to  be  pixels  belonging  to 


the  distribution  p(R,G,B)  (i.e.  fire  pixels) .  Measurements  outside 
the  ellipsoid  are  assumed  to  not  belong  to  the  distribution 
p(R,G,B)  (i.e.  not  fire  pixels). 


From  a  sample  set  of  fire  pixels  obtained  with  a  given 
sensor,  the  vector  A  is  easily  estimated  by  the  sample  mean  and  the 
independent  parameters  of  are  easily  estimated  from  the  sample 
covariance  matrix.  Once  p(R,G,B)  has  been  estimated  according  to 
this  model  and  a  threshold  T  has  been  selected,  the  classification 
results  may  be  stored  in  a  lookup  table  to  allow  high  speed  pixel 
classification. 

7.  Determination  of  Spatial  Variation 

Fires  will  exhibit  spatial  variation  in  spectral  content 
and  intensity  within  single  frames  according  to  the  function 
V(x',y',t,X) .  The  ability  to  measure  this  spatial  variation  will 
depend  on  the  distance  of  the  fire  from  the  sensor  and  the  spatial 
resolution  of  the  sensing  device.  Near  fires  will  produce  images 
with  significant  measurable  spatial  variation,  while  for  distant 
fires  most  of  this  spatial  variation  will  be  lost  as  radiance  from 
large  areas  of  the  fire  is  integrated  at  single  collection  sites  in 
the  CCD  array. 

The  spatial  variation  of  a  fire  region  R*  is 
characterized  by  the  variance  vector  {o2,a2,ab)  defined  by 


© 
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where  N  is  the  number  of  pixels  in  Rj(  (r,g,b)  are  individual  color 
pixel  measurements,  and  (r, g,b)  is  the  mean  color  vector  for  Rs. 
A  region  Rj  is  considered  to  have  appropriate  spatial  variation  for 
a  fire  if  the  corresponding  vector  (ar2,ag2,ab2)  is  in  the 
appropriate  range  for  a  fire  at  the  distance  estimated  for  the 
potential  fire  imaging  to  Rs . 

8.  Determination  of  Temporal  Variation 

Fires  will  exhibit  temporal  variation  (flicker)  in 
spectral  content  and  intensity  from  frame  to  frame  according  to  the 
function  V(x' ,y' ,t, A) .  The  ability  to  measure  this  flicker  depends 
upon  the  distance  of  the  fire  from  the  sensor  and  the  number  of 
frames  per  second  processed  by  the  imaging  system.  Near  fires  will 
produce  image  sequences  with  significant  flicker,  while  distant 
fires  will  exhibit  less  image  flicker. 

The  temporal  variation  of  an  individual  pixel  (i,j)  is 
quantified  by  the  temporal  variance  vector 
(tfrz(i/ j)  ,°g2U,  j)  ,ab2(i,  j))  /  defined  by 


ab{*>j)  =  jj  (hf  ~ 


where  N  is  the  number  of  frames  over  which  the  vector  is  computed. 


(r,-j , g,-j / bj j)  are  measured  pixel  vectors  at  location  (i,j)  over  the 
sequence  of  frames,  and  (r,- j , b,- j , g,- j)  are  the  mean  color  pixel  values 
at  (i,j)  over  the  N  frames. 


The  temporal  variation  for  a  region  R,-  is  defined  to  be 
the  mean  (ar2, ag2, ab2)  of  the  vectors  (ar2(i,  j)  ,ag2(i,  j)  ,crb2(i,  j) )  taken 
over  all  pixels  in  Rf  in  each  of  the  N  frames.  A  region  R*  is 
considered  to  have  appropriate  temporal  variation  for  a  fire  if  its 
temporal  variation  (ar2,ag2,ab2)  is  in  the  appropriate  range  for  a 
fire  at  the  distance  estimated  for  the  potential  fire  imaging  to 

R|. 


9.  False  Alarm  Source  Discrimination 

The  sources /objects /phenomena  that  cause  conventional 
detectors  to  false  alarm  or  to  be  confused  are  related  to  UV  and/or 
IR  emissions  or  reflections.  The  MVFDS,  operating  primarily  in  the 
color /visible  spectrum,  cannot  be  fooled  by  such  sources. 

During  this  study  efforts  were  made  to  confuse  the  MVFDS 
processing  algorithms  by  the  introduction  into  the  scene  of  various 
types  of  bright  lights  and  reflecting  surfaces.  As  discussed  in 
Section  IV  D,  Experimental  Results,  no  misidentif ications  were  made 
and  all  were  discriminated  as  "nonfire"  sources.  The  following  are 
some  representative  descriptions  of  how  the  MVFDS  responds  to 
various  types  of  light  sources. 

a.  Lights:  Fixed  in-place  facility  or  utility:  MVFDS 
will  identify  as  bright  objects  that  do  not  satisfy  the 
requirements  of  growth,  edge  flicker,  spectral  flicker,  or 
appropriate  color  histogram.  MVFDS  will  recognize  position 
continuity.  MVFDS  will  eliminate  from  the  standard  base  stored 
scene. 
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b.  Lights:  Vehicle,  aircraft,  AGE,  or  hand  held  lights 
moving  in  front  of  or  toward  MVFDS  detector  unit:  MVFDS  will 
recognize  as  bright  object  and  possible  growth.  MVFDS  will 
identify  as  nonfire  source  because  of  wrong  spectral  signature,  no 
edge  profile  flicker,  no  spectral  variations,  and  no  continuity 
frame  to  frame  (unless  object  is  moving  exactly  in  direct  line  of 
sight  of  detector)  .  In  the  system  approach,  where  two  or  more 
MVFDS  detection  units  are  involved,  one  detector  may  see  the  object 
growing  (moving  toward  it)  ,  but  the  other  camera  sees  it  moving 
away.  It  cannot,  therefore,  be  a  fire. 

c.  Lights  as  above,  but  of  different  colors,  rotating, 
strobe,  being  chopped  by  movements  and  fans,  etc.  Again,  the  MVFDS 
will  see  them  as  bright  lights  and  determine  their  positions.  They 
will  be  eliminated  as  fire  candidates  because  they  do  not  satisfy 
all  the  conditions  of  growth,  growth  rate,  continuity,  spatial 
flicker,  spectral  signature,  and  spectral  flicker. 

d.  Welding  torch  (flame  and  sparks 1 :  MVFDS  will 
identify  as  bright  object  but  will  take  no  dump  action  because  of 
no  growth  over  consecutive  frames,  lack  of  continuity,  small  size, 
no  high  frequency  spatial  and  spectral  flicker. 

e.  Aircraft  afterburner:  MVFDS  will  discriminate 
because  of  no  edge  flicker,  smooth  contour,  no  spectral  flicker. 
If  the  aircraft  is  in  the  distance,  the  afterburner  does  not 
portray  a  contiguous  source  in  consecutive  frames  as  seen  by  the 
MVFDS. 


f.  Matches,  cigarette  lighters:  If  moving  directly 
toward  a  MVFDS  detector,  they  will  be  identified  as  bright  objects, 
with  edge  flicker,  spectral  flicker,  but  probably  no  continuity.  It 
is  difficult  to  imagine  a  situation  where  the  small  flame  is  moved 
exactly  along  the  line  of  sight  to  the  center  of  the  detector,  and 
always  remaining  on  the  base  line  where  fire  was  first  identified. 
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The  flame  would  appear  as  a  small,  almost  point  source,  and  even  in 
the  worst  case,  the  flame  would  be  determined  to  be  too  small  to 
justify  a  dump.  If  the  flame  was  held  directly  in  front  of  the 
camera,  only  inches  away,  the  MVFDS  would  see  all  the  features  of 
fire,  including  size,  and  dump  unless  the  MVFDS  is  operating  in  a 
system  mode.  In  the  system  configuration,  the  second  or  other 
detector (s)  would  see  the  flame  as  too  small  and  possibly  moving 
away,  thus  preventing  a  suppression  dump. 

D .  EXPERIMENTAL  RESULTS 


The  algorithms  described  in  Section  IV  C  were  tested  against 
real  fire  event  data.  This  section  describes  the  results  of 
implementing  them  in  "C"  language  and  running  them  on  a  Sun  SPARC 
computer . 


1.  Tracking  Bright  Regions 

Figures  9-15  are  digitized  images  from  a  sequence  taken 
of  the  HAS  JP-4  running  fuel  fire  test  at  Tyndall  AFB  in  1985.  The 
camera  aperture  was  improperly  adjusted  to  capture  the  detailed 
spectral  or  texture  characteristics  of  the  flame.  This  sequence 
can  be  used,  however,  to  demonstrate  the  ability  of  the  algorithms 
developed  in  this  study  to  locate  bright  regions  and  to  monitor 
their  growth. 

Figure  9  is  the  first  frame  in  the  sequence.  A  small  fire  has 
started  in  the  lower  left  part  of  the  image  (assume  at  this  instant 
in  time  that  UV/IR  detectors  have  responded,  therefore  transforming 
the  MVFDS  system  operation  from  standard  to  alert  mode) .  The 
bright  pattern  of  light  entering  the  hangar  through  the  door  on  the 
right  side  of  the  image  has  persisted  without  growth  for  a 
significant  amount  of  time  before  the  UV/IR  alert  and  is  not 
considered  to  be  a  threat.  The  system  identifies  the  bright  region 
in  the  lower  left  as  a  candidate  fire  region. 
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Figure  9.  Fire  Ignition  Tine 

Frane  1 


Figure  10.  0.1  sec  Later 

Frane  2 


[Sequence  of  Inages  Beginning  with  Figure  9  of  Running 
165  gal  JP-4  Fuel  Fire  During  "HAS”  Teats  at  Tyndall  AFB,  1985] 


Figure  11.  Continued  Fire  Growth 

Frane  3 


Figure  12.  Continued  Fire  Growth 

Frane  6 


Figure  16  indicates  the  detected  growth  of  the  fire  between 
Frames  1  and  2.  In  Figure  16,  a  few  new  bright  pixels  on  the  right 
side  of  the  image  are  identified.  These  pixels  are  due  to  random 
fluctuations  in  the  edges  surrounding  the  light  passing  through  the 
door  and  can  be  rejected  automatically  as  threats  because  of  their 
small  "point”  size. 

Figure  17  indicates  the  detected  growth  of  the  fire  between 
Frames  2  and  3.  Since  the  estimated  growth  is  based  only  on 
intensity,  a  region  made  up  of  the  reflection  of  the  fire  from  the 
airplane  is  detected  as  a  growth  region.  Figure  18  tracks  the 
growth  of  the  fire  between  Frames  3  and  4  and  Figure  19  tracks  the 
growth  of  the  fire  between  Frames  4  and  5.  In  Frame  6,  a  second 
fire  starts  in  the  lower  right  part  of  the  image  and  its  presence 
along  with  the  continued  growth  of  the  first  fire  are  indicated  in 
Figure  20.  In  Frame  7,  Figure  15,  the  second  fire  grows  and 
suppressant  is  dumped  from  the  upper  left.  These  events  are 
detected  by  the  algorithm  as  indicated  in  Figure  21. 

2.  Identifying  Fires  by  Spectral  Content 

Figures  22-37  illustrate  the  capability  of  the  algorithm 
described  in  Section  IV  C-6  to  locate  fire  pixels  using  only  their 
spectral  signature.  Figures  22,  24,  26,  28,  30,  32,  34,  and  36  are 
eight  color  frames  taken  of  a  growing  JP-4  running  fuel  fire  at 
Tyndall  AFB.  Each  frame  contains  several  bright  regions  in 
addition  to  the  fire.  Figures  23,  25,  27,  29,  31,  33,  35,  and  37 
indicate  in  white  the  pixels  that  the  algorithm  classified  as  fire 
using  only  local  single  pixel  color  information.  These  figures 
show  that  this  technique  is  very  accurate  in  identifying  only  fire 
pixels  in  the  presence  of  other  bright  stimuli.  In  addition,  these 
figures  show  that  the  method  can  capture  fine  spatial  details  in 
the  fire  structure. 
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Figure  16 .  Detected  Growth  Between 
Frames  1  and  2 


Figure  17 .  Growth  Between 
Frames  2  and  3 


Figure  18.  Growth  Between 
Frames  3  and  4 


Figure  19.  Grwoth  Between 
Frames  4  and  S 
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Figure  20.  Second  Fire  Ignited  in  Lower  Right 


Figure  21.  Second  Fire  Growth  in  0.1  sec 
(note  halon  release  upper  left) 


53 


Figure  22.  35mm  Color  Still  Photo  Figure  23.  Result  of  Classification 

at  Fire  Ignition  in  Algoritha  Processing 

"HAS”  Test  Pixels  for  "Fire" 

Signature  (note  all  other 
brightob j ect  sel iminated ) 


Figure  24.  Color  Photo  0.3  sec  Later  Figure  25.  Results  of  Algoritha 

Processing 


Figure  26 .  Color  Photo  of  Fire 
after  0.6  sec 
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Figure  28.  Next  Photo  0.9  sec 
After  Fire  Start 


Figure  27 .  Result  of  Algorithm 

Processing  for  "Fire" 
Pixels 


Figure  29 .  Results  of  Algorithm 
Processing  of  Pixels 


Figure  32.  Next  Color  Photo  1.5  sec  Figure  33.  Results  of  Algorithm 
after  Fire  Start  Processing  of  Pixels 
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Figure  34.  Next  Color  Still  Photo 

1.8  sec  after  Fire  Start 


Figure  35.  Continued  Success  of 
Algorithm  Processing 
for  Fire  Pixels 


Figure  32.  Next  Color  Photo  2.1  sec  Figure  33.  Continued  Results  of 
®fi@r  Fire  Start  Algorithm  Processing 
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Figures  38-41  further  illustrate  the  ability  of  the  spectral 
classification  algorithm  to  identify  fire  regions  in  the  presence 
of  potential  false  alarm  sources.  Figures  38  and  40  are  frames 
containing  a  7-inch  fire  at  a  distance  of  50  feet  from  the  camera, 
a  floodlight  at  8  feet  from  the  camera,  and  a  highly  reflective 
card  containing  many  bright  colors  of  different  patterns  intending 
to  confuse  the  algorithm.  Figures  39  and  41  indicate  in  white  the 
pixels  that  the  algorithm  classified  as  fire  using  only  local 
single  pixel  color  information.  These  figures  demonstrate  that  the 
algorithm  can  locate  small  fires  in  the  presence  of  false-alarm 
stimuli . 

3.  Identifying  Fires  by  Spatial  Variation 

Figures  42  and  43  demonstrate  the  ability  of  the 
algorithm  to  measure  spatial  variation  (texture)  within  a  single 
frame  of  a  fire  sequence.  Figure  42  is  an  image  of  a  16-inch  pan 
fire  19-inches  from  the  camera.  Intensity  and  color  spatial 
variation  within  the  fire  is  clearly  evident.  Figure  43  indicates 
in  white  local  image  regions  that  exhibit  large  spatial  variations. 
These  figures  demonstrate  that  spatial  variation  is  a  useful  cue 
for  fire  identification. 

4.  Identifying  Fires  by  Temporal  Variation 

The  ability  of  the  algorithm  of  Section  IV  C-8  to  measure 
temporal  variation  (flicker)  is  shown  in  Figures  44-48.  Figures 
44-47  are  four  consecutive  frames  taken  of  a  16  inch  pan  fire  19 
inches  from  the  camera.  A  small  rectangle  of  approximately  the 
same  color  as  the  fire  has  been  superimposed  near  the  upper  left 
hand  corner  of  the  image. 

Figure  48  indicates  in  white  the  pixels  that  exhibit 
significant  temporal  flicker.  Nearly  all  the  fire  pixels  that  are 
within  the  dynamic  range  of  the  imaging  system  are  marked.  As 


Figure  38.  7-Inch  JP-4  Pan  Fire  with 
Intense  Strobe  Light  and 
Color  Reflector  in  Scene 


Figure  39.  Results  of  Classifying 
Pixels  with  Algorithm 
to  Discriminate  False 
Alarm  Sources 


Figure  40.  7-inch  JP-4  Pan  Fire  with  Figure  41.  Continued  Results  of 
Moving  Bright  Xenon  Light  Classifying  "Fire" 

and  Reflecting  Color  Stripes  from  "Other"  Pixels 
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Figure  42.  Color  Frame  of  16-inch  JP-4  Pan  Fire  Showing  Detailed 
Spectral  structure  and  Variance  Pixel-to-Pixel 


Figure  43.  Results  of  Processing  above  Digitised  Frane  for  Areas 

Showing  Large  Spectral  Variations  (this  demonstrates  that 
spatial  variation  is  a  very  useful  fire  discriminator) 
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Figure  44.  16-inch  JP-4  Pan  Fire  at  Figure  45.  Frame  2  1/30  sec 

19-inches  -  Frame  1  of  Sequence  after  Frame  1 

Taken  with  CCD  Color  Camera 


Figure  46.  Frame  3  1/30  sec  after  Frame  2  Figure  47. 


Frame  4  1/30  sec 
after  Frame  3 


Figure  48. 


Results  of  Algorithm  Processing  to  Exhibit  Pixels  with  Large  Temporal 
Spectral  Variations 


expected,  it  can  be  concluded  that  temporal  variation  is  an 
important  fire  characteristic  that  can  be  used  for  discrimination. 

E.  MVFDS  SYSTEM  HARDWARE 

The  system  consists  of  discrete  UV  and  IR  detectors,  video 
cameras,  real  time  video  image  digitizing  interfaces,  a 
microcomputer  image  processor,  and  outputs  to  initiate  alarms  and 
suppressant  activations.  Figure  49  is  a  block  diagram  of  a 
prototype  hardware  configuration. 

One  or  more  color  video  cameras  are  used  for  the  videoimaging 
fire  detectors.  The  camera's  RGB  (red, green, blue)  analog  output 
signals  are  converted  into  digital  computer  data  that  is  stored  in 
video  memory  to  accumulate  a  complete  picture  or  frame.  Each 
"frame"  can  be  captured  upon  command  and  transferred  into  the 
microcomputer  using  a  high  speed  real-time  video  processor  commonly 
known  as  a  frame  grabber  card  (FGC) .  The  camera  video  output  is 
connected  to  a  FGC  which  is  plugged  into  an  expansion  connector 
slot  on  the  microcomputer  main  printed  circuit  board.  The 
microcomputer  processes  the  spatial  and  spectral  information 
obtained  from  fire  emissions  with  the  color  video  cameras  to 
effectively  discriminate  from  nonfire  or  benign  fire  emission 
sources. 

The  following  description  of  the  system's  functions  is  based 
upon  currently  available  commercial  hardware.  Video  cameras, 
frame  grabber  cards,  and  microcomputer  hardware  are  identified  plus 
the  essential  operating  software  required  to  develop,  refine,  and 
implement  the  computerized  machine  vision  fire  detection 
technology. 

1.  Color  Video  Camera 

The  primary  detector  used  in  the  system  is  a  standard 
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solid  state  Charge  Coupled  Device  (CCD)  color  video  camera  that 
produces  three  primary  color  signal  outputs,  red,  green,  and  blue 
(RGB) ,  which  can  be  recombined  in  a  display  device  to  recreate  the 
multitude  of  original  colors  in  the  image.  Only  cameras  with 
rugged  solid  state  silicon  CCD  imaging  devices  are  considered  to 
meet  the  reasonably  high  levels  of  shock  and  vibration  that  occur 
in  military  operational  environments.  The  CCD  imaging  device 
consists  of  a  rectangular  solid  state  light  detector  array  composed 
of  individual  picture  elements  called  pixels.  Each  pixel  has  a 
defined  horizontal  (HORZ)  column  and  vertical  (VERT)  row  coordinate 
(address)  in  the  array  by  which  it  can  be  accessed. 

The  color  image  focused  upon  the  detector  surface  is 
separated  into  three  primary  colors,  red,  green,  and  blue,  by 
vertical  RGB  color  stripe  filters  placed  over  each  set  of  three 
horizontally  adjacent  pixels.  The  impinging  light  level/color 
information  acquired  by  all  the  array  pixels  is  scanned 
sequentially  by  the  camera  internal  electronics  to  produce  serial 
video  analog  voltage  output  signals  for  each  of  the  three  primary 
colors,  RGB.  Presently  available  CCD  color  cameras  have  very  good 
low  light  level  resolution,  contrast,  and  wide  dynamic  range  which 
is  necessary  in  applications  having  highly  variable  lighting 
conditions  (such  as  aircraft  shelters).  The  CCD  camera's  high 
performance  and  very  small  size  are  very  important  factors  in 
configuring  the  MVFDS. 

The  CCD  cameras  are  available  in  a  variety  of  cost 
effective  horizontal  and  vertical  pixel  array  resolutions  from  a 
low  resolution  128-  x  128-pixel  array,  to  a  512-  x  512-pixel  medium 
resolution,  and  a  748-  x  484-pixel  medium  high  resolution.  As 
shown  in  Figure  50,  with  the  latter  resolution  of  748-  x  484-pixel 
array,  and  a  90-degree  f ield-of-view  lens,  fires  near  1  ft2  in  size 
at  about  100  feet  distance  can  be  resolved  (it  requires  at  least  9 
contiguous  pixels  to  be  able  to  perform  the  all  the 
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resolution 


pixel  lens 

array  d  =  distance 

n  =  748 


discrimination  functions) . 

In  Figure  49  the  image  of  a  fire,  represented  by  the 
triangle,  is  focused  by  the  camera  lens  upon  the  CCD  imaging  device 
pixel  array  which  translates  the  light  level  at  each  pixel  into  an 
analog  voltage.  The  camera  electronics  provides  precision  clock 
controlled  timing  and  sweep  drive  for  HORZ  and  VERT  scanning  of  the 
CCD  pixel  array.  This  produces  a  serial  stream  of  pixel  data  for 
each  of  the  three  colors.  Precise  horizontal  and  vertical 
synchronization  (HORZ  SYNC  and  VERT  SYNC)  are  outputted  to  define 
the  format  of  each  complete  frame.  Each  complete  video  frame  is 
outputted  at  the  standard  60  noninterlaced  frames  per  second  (two 
interlaced  frames  in  1/30  second) .  The  electronics  also  provides 
control  of  exposure  integration  time  to  accommodate  for  varying 
lighting  conditions.  The  scanning  process  is  described  in  Figure 
51. 

The  VERT  SYNC  pulse  signals  the  beginning  of  scanning  one 
video  frame  starting  at  the  top  left  corner  of  the  image  captured 
on  the  CCD  pixel  array.  The  HORZ  SYNC  pulse  initiates  the  scan 
from  the  right  across  the  top  row  of  pixels  and,  when  it  reaches 
the  right  side,  another  HORZ  SYNC  pulse  is  produced  which  starts 
the  next  scan  line.  This  is  repeated,  each  time  incrementing  to 
the  next  line  below,  until  it  reaches  the  bottom  of  the  frame,  at 
which  time  another  VERT  SYNC  pulse  returns  the  scan  to  the  top  of 
the  screen  to  begin  the  next  video  frame.  This  new  frame  is 
interlaced,  starting  at  the  middle  of  the  top  line. 

The  color  camera  produces  the  video  industry  standard 
three  primary  color  analog  outputs  (RGB) .  It  also  outputs 
horizontal  and  vertical  synchronization  (SYNC)  signals  which  output 
separately  or  combined  in  the  industry  standard  NTSC  or  Y-C 
formatted  video  output  signal.  The  three  primary  color  RGB  levels 
when  recombined  in  triad  combinations  can  recreate  all  the  colors. 
Figure  52  shows  the  color  cube  domain. 
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Figure  51.  NTSC  Scanning  and  Timing  Drawings 
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Interlaced  scanning  pattern  (raster) 

— — - even  frame  scan  lines 

- odd  frame  scan  lines 
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Figure  52.  RGB  Color  Cube  Domain 
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resulting  in  every  perceivable  color.  Th 
includes  all  the  characteristic  fire  colors. 


2.  UV  and  IR  Detectors 

In  addition  to  the  primary  color  camera,  discrete  UV  and 
IR  detectors  operating  outside  the  visible  spectrum  are  used  to 
provide  early  warning  of  significant  fire  emission  spectral 
attributes.  These  have  been  discussed  previously  herein.  However, 
these  are  operated  at  high  sensitivity.  When  both  have  registered 
the  presence  of  radiation,  the  MVFDS  is  switched  from  standby 
processing  to  a  fast  event  processing  mode.  These  detectors  are 
available  from  several  sources  and  are  inexpensive  (typically  less 
that  $100)  compared  to  the  cost  of  complete  UV/IR  commercial 
fire/flame  detector  units  (typically  $1 , 500-$2 , 000) . 

3 .  Computer  Image  Processor  Components 

a.  Video  Frame  Grabber  Card 

The  FGC  is  the  key  interface  to  translate  the  color 
video  camera  analog  RGB  output  signals  into  digitized  data.  The 
microcomputer  can  instruct  the  FGC  to  capture  a  video  frame  output 
from  the  video  camera,  digitize,  and  store  the  frame  in  the  high 
speed  video  memory.  The  digitally  stored  frame  image  is  then 
accessible  to  the  computer  for  processing  via  the  I/O  Bus  interface 
connector.  The  microcomputer  can  capture  frames  at  30  or  so  frames 
per  second.  Tests  performed  during  this  study  with  an  R&D  system 
on  scenes  from  video  taped  fire  tests  performed  in  a  hangar 
environment  suggests  that  frame  capturing  and  processing  once  every 
0.1  seconds,  or  10  frames  per  second,  is  sufficient  to  follow  a 
fire  growth  and  determine  other  features.  Preliminary  development 
indicates  that  all  the  essential  algorithms  can  be  processed  at 
this  rate. 

b.  Microcomputer  Mother  Board 

The  MVFDS  main  control  80386  type  microcomputer  is  on  the 


mother  board  which  has  a  set  of  several  expansion  I/O  Bus  connector 
slots.  The  connectors  accept  the  FGC  and  I/O  card  to  interface 
with  the  detectors  and  provide  output  to  activate  alarms  and 
suppressors  through  a  fire  control  panel,  if  desired. 

c.  Software/Firmware 

The  hardware  requires  a  software  program  to  operate 
the  microcomputer  and  control  the  system.  The  microcomputer 
requires  operating  system  software  that  initiates  operation  of  the 
microcomputer  using  a  "Basic  Input  Output  System"  (BIOS)  low-level 
machine  language  permanently  stored  in  a  "Read  Only  Memory"  (ROM) 
integrated  circuit.  The  software  performs  microcomputer  and  system 
initialization,  progresses  to  a  self  testing  procedure,  and  begins 
running  the  Fire  Detection  Program  (FDP)  with  periodic  self  test 
procedures  run  to  determine  operational  status.  The  majority  of 
the  FDP  will  be  programmed  in  a  high  level  language  (such  as  "C") 
except  for  any  speed  sensitive  subroutine  that  will  be  written  in 
high  speed  machine  language.  The  machine  level  language  FDP 
program  will  be  stored  in  ROM. 

4 .  Output  Devices 

The  input  and  output  (I/O)  activity  of  the  microcomputer 
operating  under  control  of  the  FDP  uses  the  I/O  card  to  accept  the 
UV  and  IR  detector  outputs  and  send  the  data  to  the  microcomputer 
for  action.  Fire  alarm  and  suppressor  activation  decisions  from 
the  microcomputer  are  conditioned  to  interface  with  the  alarm  and 
suppressor  power  drivers  and  activation  devices,  usually  found  in 
the  fire  control  panel. 

Alarm  driver  outputs  and  alarm  status  signals  to  facility 
alarms  and  to  the  local  fire  authority,  and  high  power  drivers  to 
suppressant  systems  (or  controller  panels) ,  can  be  provided  on  the 
I/O  card.  They  can  also  be  provided  in  a  separate  enclosure,  as 
direct  interfaces  to  existing  communications  control  panels. 
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5. 


Power  Supply 


Highly  regulated  low  voltages  are  required  for  the 
following  system  components:  the  microcomputer,  expansion  boards, 
camera  electronics,  and  output  driver  power.  These  are  provided  by 
a  high  efficiency  switching  power  supply  operating  off  local  power 
mains,  or  by  an  uninterruptable  power  supply  (UPS)  as  required. 

6.  Enclosure  and  Connectors 

The  whole  MVFDS  system  can  be  packaged  in  a  single 
enclosure  sealed  from  the  environment  with  sealed  connectors/cables 
provided  for  power  and  I/O's.  The  I/O  would  also  provide  for 
multiple  secondary  camera  detector  units  to  be  placed  strategically 
in  large  hangar  applications  or  other  facility  application  that 
would  benefit  from  more  than  one  detector  unit  coverage. 

7 .  Summary 

The  hardware  necessary  to  implement  the  MVFDS  exists  in 
suitable  low  cost,  low  power,  small  size  and  reliable  components 
available  to  package  in  a  single  unit  for  R&D,  field  testing,  an 
operational  installation  in  fixed  ground  based  environments. 
Further  refinement  and  simplification  of  the  hardware/software 
system  can  reduce  size,  power  and  ruggedness  for  future  production 
systems  and  broader  applications. 

The  miniaturized  camera  sizes  are  indicated  by  the 
outline  of  the  Cohu  model  6815  camera.  The  control  box  electronics 
of  the  camera  contain  much  more  electronics  than  needed,  such  as 
NTSC  encoding,  auto  exposure  lens  control,  power  regulators,  etc. 
These  can  be  eliminated.  Therefore,  its  size  can  be  reduced  to 
about  one  fourth  of  the  present  size.  The  FGC  cards  for  IBM 
compatible  computers  are  generally  implemented  on  the  5-inch  high 
by  13-inch  long  AT  (Advanced  Technology)  card  formats.  They  too 


71 


have  much  more  circuitry  than  necessary,  namely  NTSC 
encoders/decoders,  special  effects,  genlock,  etc.  Eliminating 
these  unnecessary  circuits  can  reduce  the  size  by  a  minimum  of  one 
half.  The  proliferation  of  low  cost  PC-AT  miniaturized  computer 
mother  boards  using  VLSI  chip  sets  with  sizes  only  one  fourth  of 
the  11-inch  x  13-inch  standard  size  boards,  along  with  reduced  size 
peripheral  boards  and  power  supplies,  will  yield  a  very  small  MVFDS 
system  footprint. 

The  extremely  large  numbers  of  systems  in  use,  and  the 
manufacturers  that  produce  them,  dictate  using  main  stream 
technology  to  guarantee  a  no-risk  MVFDS  with  increasing  hardware 
reliability  and  increasing  capability  and  versatility  for  systems 
with  future  expansion  and  upgrades. 

The  primary  MVFDS  system  with  one  set  of  sensors,  frame 
grabber,  computer,  I/O,  and  power  supply,  can  be  engineered  to  fit 
within  the  estimated  envelope  shown  in  Figure  53. 
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Figure  53.  MVFDS  R&D  Prototype  (not  production  scale) 
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SECTION  V 


UTILIZATION  AND  APPLICATIONS  OF  MVFDS 


A.  COMPARISON  OF  MVFDS  WITH  CONVENTIONAL  FIRE/FLAME  DETECTORS 

The  basic  difference  between  conventional  detectors  (CDs)  and 
the  MVFDS  is  in  the  quantity  and  quality  of  the  information 
generated  by  each.  In  the  case  of  the  CD,  the  only  information 
obtained  is  intensity  levels  of  UV  and/or  IR  radiation  at  some 
specific  wavelength  regions.  The  detector  does  not  know  the 
direction,  distance,  location,  size  or  nature  of  the  source  of  the 
radiation.  It  only  knows  that  it  is  present.  On  the  other  hand, 
the  MVFDS  operates  in  the  visible  region  (like  a  human  being),  and 
knows  the  source  of  the  light,  location,  size,  distance,  spectral 
color  features,  movement,  association  to  other  objects  and,  of  most 
importance,  that  the  nature  of  the  light  emitting  object /phenomena 
is  indeed  a  fire.  CDs  cannot  accomplish  any  of  the  former  and  are 
unsure  of  the  precise  nature  of  the  emitting  object.  There  are, 
therefore,  major  difference  between  the  two  detection  concepts. 

One  characteristic  of  some  UV/IR  detectors  that  utilize  fast 
response  UV  vacuum  tubes  and  IR  detectors  such  as  PbSe,  is  their 
ability  to  detect  the  presence  of  very  small  levels  of  UV  and  IR 
within  specific  wavelength  regions  (e.g.  190  -  300  nanometers,  and 
4.3  micrometers).  This  sensitivity  allows  them  to  respond  very 
rapidly,  almost  instantaneously,  to  fire  ignition.  However,  this 
ability  is  also  a  detriment  to  reliable  operation  because  the 
detector  becomes  more  susceptible  to  false  alarming  to  the  presence 
of  other  UV/IR  source(s).  However,  in  conjunction  with  the  MVFDS 
visible  intelligence,  machine  vision  operation,  the  presence  of 
very  small  quantities  of  UV  and  IR  can  be  used  advantageously  as  a 
"switch"  to  transfer  the  MVFDS  from  standby  mode  to  fast  "alert" 
processing  mode,  and  as  an  "AND"  in  the  logic  decision  process  of 
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fire  determination.  The  MVFDS  can  either  use  its  own  very 
sensitive  UV/IR  detectors  (which  are  inexpensive  and  simple 
compared  to  CDs) ,  or  use  the  outputs  of  existing,  already  installed 
UV/IR  detectors  for  these  purposes. 

Some  specific  differences  between  CDs  and  the  MVFDS  are  as 
discussed  below: 

1 .  Fire  Detection 


Both  detect  fires  but  by  different  techniques.  The  CD  relies 
on  an  indirect  technique  whereby  measurement  of  UV/IR  above  a 
certain  preset  threshold  intensity  level  is  equated  with  a  fire  of 
some  predetermined/ calibrated  size  at  some  maximum  distance, 
although  the  CD  does  not  know  anything  about  the  nature  and/or 
properties  of  the  emitting  source(s).  The  MVFDS  uses  UV/IR  to 
"suggest"  the  presence  of  fire  and  then  uses  visible  light  input 
and  computer  processing  to  reach  an  "intelligent  decision,  just  as 
a  human  being  uses  his  eyes  as  input  and  his  brain  as  the 
information  processor.  The  former  CD  technique  can  be  labeled 
"indirect"  as  opposed  to  the  MVFDS  direct  technique. 

2 •  Fire  Location 

CDs  cannot  determine  where  the  emitting  source  is  located 
while  the  MVFDS  knows  its  distance,  size,  and  position  relative  to 
other  objects  within  the  f ield-of-view. 

3 •  Fire  Size 

As  stated  above,  CDs  only  know  that  some  threshold  intensity 
level  of  UV/IR  has  been  detected.  They  are  either  preset  in  the 
factory  or  calibrated  at  installation  to  be  able  to  detect  a  fire 
of  some  size  at  some  distance.  Usually  a  pan  fire  of  some  size, 
say  10-foot  x  10-foot,  is  set  at  some  distance,  say  100  feet,  and 
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the  detector  electronics  adjusted  until  it  detects  the  fire  within 
some  maximum  time  period  (usually  5  seconds  or  so,  where  time  zero 
occurs  when  the  detector  is  exposed  to  an  already  burning  10-foot 
x  10-foot  pan  fire) .  This  only  assures  that  a  CD  will  see  this 
level  of  intensity  of  UV/IR.  A  small  fire,  however,  of  less 
distance  away  from  the  detector,  will  be  interpreted  as  being  a 
large  fire  at  some  greater  distance.  This  is  because  of  the  1/R2 
dependence  and  that  the  source  is  omnidirectional. 

The  MVFDS,  however,  knows  the  distance  of  the  source,  the 
number  of  pixels  occupied  by  the  source,  the  size  of  these  pixels, 
and,  therefore  the  actual  size  of  the  fire  event.  It  can  monitor 
the  fire's  growth  until  it  reaches  a  size  requiring  ar.  alarm  and/or 
an  automatic  release  of  suppressant.  This  provides  for 
instantaneous  action  instead  of  3-5  seconds  or  so  delayed  action  as 
required  of  CDs  today. 

4 .  Time  Required  to  Identify  Fire  Event 


As  inferred  above,  CDs  can  detect  UV/IR  from  a  fire 
instantaneously.  But  to  reduce  false  alarm  problems,  they  are 
usually  set  to  respond  in  3-5  seconds  after  the  fire  reaches  the 
threshold  size  for  detection  (which  can  be  10-foot  x  10-foot  size 
at  150  feet  distance) .  CDs  normally  operate  in  this  3-5  seconds 
range  and  are,  therefore,  not  reliable  to  detect  small  fires  in 
very  short  times  (tradeoff  of  fire  sensitivity  to  false  alarm 
susceptibility) .  The  MVFDS  uses  the  UV/IR  to  instantly  detect 
"possible"  fire  presence  and  then  uses  the  visible  to  locate  and 
then  analyze  all  the  characteristics  of  the  suspect  event/object 
before  making  a  fire  decision.  From  the  tests  made  in  this  study, 
it  appears  that  the  MVFDS  can  identify /discriminate  a  fire  of  about 
1-2  ft2  in  size  at  a  distance  of  100  feet,  and  that  4  or  5  frames 
of  processing  is  all  that  is  required  to  determine  all  the 
characteristics  of  the  fire  event,  including  its  size  and  location. 


At  one  frame  every  100  milliseconds,  this  time  period  would  be 
about  0.4  to  0.5  seconds . 

5 .  Immunity  to  False  Alarms 

CDs  respond  to  all  sources  of  UV  and/or  IR  radiation  that  are 
within  the  wavelength  sensitivities  of  the  detector.  CDs  are  thus 
susceptible  to  false  alarming  to  the  presence  of  one  or  more  of  the 
following  type  of  objects /phenomena:  lights  of  many  varieties, 
heaters,  aircraft  ground  equipments  (AGE) ,  tools,  electric 
discharges,  lightning,  welding,  aircraft  engines  from  start  through 
afterburner,  heat  from  engine  exhausts,  x-rays,  and  many  other 
items  discussed  in  detail  in  Reference  1.  Although  not  a  UV  or  IR 
source.  X-ray  machines  (NDI  tools)  can  set  off  any  UV  vacuum  tube 
detector  because  it  operates  as  an  ionization  chamber  such  as  a 
Geiger  Mueller  Counter.  There  is  some  evidence,  also,  that  such 
NDI  devices  produce  high  levels  of  EMI,  thus  affecting  the 
electronics  of  CDs  that  are  not  EMI  proof. 

Although  the  MVFDS  may  be  set  to  be  very  sensitive  to  UV  and 
IR,  it  does  not  respond  to  them  as  fire  sources.  It  discriminates 
fire  from  possible  false  alarm  objects  in  the  visible  by  discerning 
their  spatial,  temporal  and  spectral  variations,  as  well  as  other 
features  that  are  unique  to  fire  and  not  associated  with  others  or 
combination  of  sources  so  far  identified. 

6.  Physical  Features 

The  MVFDS  detector  units  are  smaller  than  those  of  CDs  and  can 
be  mounted  in  the  same  manner  and  configuration.  The  first 
generation  MVFDS  engineering  model  will  probably  be  about  13-inch 
x  8  1/2-inch  x  4-inch,  with  the  capability  of  multiple  detector 
unit  inputs.  The  first  production  model  would  be  smaller  (about  6- 
inch  x  8-inch  x  4-inch) . 


77 


The  MVFDS  can  come  in  two  configurations:  a  single  detector 
unit  with  computer  electronics  in  one  container,  and  multiple 
detector  units  feeding  into  one  computer  electronics  box,  thus 
enabling  small  area  fire  surveillance  as  well  as  large  hangar 
surveillance  such  as  B-2  hangars.  The  number  of  CDs  required  to 
perform  a  system  application  such  as  a  hangar  is  larger  than  that 
required  by  the  MVFDS. 

Although  CDs  are  not  designed  to  meet  stringent  military 
design  and  performance  standards,  they  can  be  designed  to  do  so. 
There  may,  however,  be  some  costs  associated  with  such  redesign  and 
testing  efforts.  The  MVFDS,  as  a  new  device,  should  be  designed 
initially  to  withstand  the  environments  specified  in  Mil-Std-810D 
and  Mil-Std-461/462 ,  and  have  the  necessary  MTBF  and  system 
reliability  to  assure  adequate  fire  protection  of  mission  essential 
weapon  systems  and  other  assets. 

7.  Producibility/Technical  Risk 

CDs  are  obviously  being  manufactured  and  their  use  has  been 
accepted  for  many  types  of  fire  detection  applications.  They  are 
designed  for  commercial  applications  and  there  is  no  technical  risk 
in  the  component  technology  or  technical  approach  employed.  Like 
CDs,  every  component  required  in  the  MVFDS  is  now  being 
manufactured  and  is  available  within  the  framework  of  the  video 
camera  and  PC  computer  industry.  There  exist  a  multitude  of 
manufacturers  of  these  and  related  hardware  items  and  the 
technology  is  being  advanced  daily  while  the  costs  continually 
decrease. 

B.  COMPATIBILITY  WITH  CURRENTLY  INSTALLED  FIRE  PROTECTION  SYSTEMS 

Many  new  fire  protection  systems  have  been  installed,  either 
in  new  construction  or  as  replacements/upgrades  to  old,  less 
capable  technology.  The  detection  units,  although  they  are  much 


more  reliable  than  older  single  channel  detectors,  still  experience 
false  alarms  and  false  activations.  These  installed  fire 
protection  systems  could  be  made  more  effective  and  certainly  more 
reliable  if  the  MVFDS  was  integrated  into  their  configurations. 
CDs  with  dual  UV/IR  detectors  could  be  used  as  input  signals  to  the 
MVFDS  if  they  were  set  at  very  sensitive  levels  to  act  as  alert 
switches  to  the  MVFDS  camera/computer  system  to  changes  from 
standby  mode  to  fast  alert  processing  mode.  For  new  construction 
and  certain  military  applications,  the  MVFDS  should  be  considered 
on  its  own  merit  depending  upon  the  performance  requirements  and 
the  problems  needing  solutions. 

As  a  retrofit,  the  MVFDS  would  have  no  problems.  It  could  be 
easily  interfaced  with  current  installed  fire  control  and 
communication  panels.  The  outputs  of  the  MVFDS  could  be  made  to  be 
simple  electrical  signal  inputs  to  the  exiting  panels. 

There  are  many  advantages  of  integrating  the  MVFDS  with 
installed  CD  systems.  This  includes,  of  course,  increased  coverage 
of  fire  sources  of  all  sizes  at  all  locations  within  very  short 
time  periods,  accuracy  of  fire  location,  size,  and  threat,  and 
elimination  of  false  alarms  and  false  dumps  of  suppressant. 
Another  advantage  that  may  be  considered  is  the  replacement  of  any 
manned  TV  monitoring  system  employed  for  fire  observation.  The 
automated  capability  of  the  MVFDS  far  exceeds  the  ability  of  a 
manned  TV  monitoring  system,  saves  money,  and  provides  the  system 
reliability  and  performance  levels  required  to  protect  valuable 
assets.  This  is  especially  true  of  those  situations  where  very 
rapid  and  reliable  fire  detection  is  required  to  enable  suppressant 
application  within  seconds.  However,  if  desired,  the  MVFDS  could 
also  act  in  the  automatic  and  manned  TV  monitoring  mode,  as  the 
video  output  (input  to  the  computer)  can  also  be  used  for  other 
purposes. 
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SECTION  VI 


CONCLUSIONS 


The  major  conclusion  of  this  Phase  I  study  is  that  it  is 
technically  feasible  to  develop  a  machine  vision  fire  detector 
system,  in  a  timely  manner,  at  reasonable  cost,  and  with  no 
technical  risk.  It  has  also  been  demonstrated  that  there  are 
certain  unique  characteristics  that  distinguish  fire  from  possible 
false  alarm  sources,  that  can  be  accurately,  rapidly,  and 
unequivocally  determined  with  the  algorithms/ software  developed  in 
this  study  and  implemented  with  the  computer  and  camera  hardware 
identified  in  this  study.  These  MVFDS  capabilities  provide  for 
major  increase  in  fire  detector  immunity  to  false  alarms,  greater 
fire  protection  reliability  and,  for  the  first  time,  very  high 
resolution/ identification  of  small  fire  events  at  large  distances. 
The  following  are  supporting  conclusions. 

1.  The  presence  of  UV  and  IR  at  wavelengths  of  190-300 
nanometers  and  4.3  micrometers,  respectively,  night  indicate  the 
presence  of  fire,  but  not  unequivocally.  Such  precursor 
information  can  be  used  by  the  MVFDS  to  "switch"  from  standard 
processing  mode  into  a  fast  "alert  mode"  and  for  selective  image 
processing.  In  addition,  the  presence  of  UV  and  IR  can  also 
function  as  an  "AND"  in  the  logic  decision  tree.  As  the  MVFDS  is 
further  developed,  the  use  of  this  "AND"  and  precursor  signals  may 
prove  to  be  redundant. 

2.  Bright  areas  within  a  scene  can  be  identified,  their 
positions  located,  and  tracked.  Further,  the  bright  areas  that  are 
present  scene  to  scene  when  there  is  no  indication  of  the  presence 
of  UV  and  IR  can  be  eliminated  from  the  stored  base  scene,  thus 
reducing  processing. 
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3.  Bright  areas  identified  when  UV  and  IR  are  present  can  be 
further  discriminated  by  whether  their  spectral  color  falls  within 
the  selected  red,  green,  and  blue  color  thresholds  indicative  of 
fire. 


4.  Scenes  can  be  captured,  digitized,  and  stored  at  rates  of 
at  least  30  frames  per  second. 

5.  The  growth  and  growth  rate  of  a  bright  area  can  be 
determined  by  frame  subtraction  of  new  frames  from  a  base  stored 
frame.  Growth  is  an  obvious  characteristic  of  all  fire  events. 

6.  The  edge  profile  of  bright  areas  can  be  determined  and 
measured  frame-to-frame  for  time  variations  ("flicker") ,  which  is 
a  characteristic  of  fire. 

7 .  The  spectral  variations  from  pixel  to  pixel  in  the  same 
image,  and  the  spectral  variations  of  the  same  pixels  frame-to- 
frame  can  be  determined.  The  magnitude  of  these  variations  have 
been  shown  to  be  unique  to  fire  and  not  associated  with  bright 
light  sources,  objects,  or  phenomena  that  may  be  classified  as 
potential  false  alarm  sources. 

8.  The  size  and  position  of  the  bright  object,  fire  event, 
can  be  determined  either  through  multiple  detector  triangulation  or 
because  its  distance  from  a  single  camera  unit  is  known  by  the  line 
scan  number  that  identifies  its  base  on  the  floor.  Each  pixel 
dimension  is  then  known  because  the  number  of  pixels  in  the 
horizontal  and  vertical  scan  planes  are  known.  The  MVFDS  can, 
therefore,  actually  determine  fire  event  size  and  position  in  the 
process  of  deciding  when  and  if  to  release  suppressant.  No  other 
detector  technique  provides  such  direct  information. 

9.  The  algorithms  developed  in  Phase  I  have  successfully 
discriminated  bright  moving  lights,  colored  flickering  lights. 
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rotating  color  lights,  strobe  lights,  highly  reflective  striped 
color  reflectors,  solar  spectral  reflection  surfaces,  and  other 
objects  and  phenomena  from  fire  events. 

10.  It  was  possible  to  obtain  and  process  the  data  at  0.1- 
second  intervals  using  PC  types  of  commercial  microcomputer 
hardware . 


11.  The  camera  hardware  and  computer  component  hardware 
required  to  implement  the  MVFDS  capabilities  exists  as  "off-the- 
shelf"  items.  There  is  no  technical  risk  in  the  full  development 
of  the  MVFDS.  The  small  quantity  costs  for  a  first  generation 
device  are  in  the  same  realm  as  present  conventional 
detectors/ logic  controllers,  and  will  decrease  in  time  with  the 
advance  of  technology  and  larger  quantity  production  runs. 

12.  The  major  algorithms  for  the  MVFDS  image  processing  have 
been  defined  in  the  Phase  I  effort,  as  well  as  the  fire  and  false 
alarm  event  models.  They  are  ready  for  refinement,  further 
testing,  and  coding  to  run  efficiently  on  the  prototype  hardware. 

13.  The  concepts  developed  in  this  study  can  make  a  major 
contribution  to  fire  protection  technology  and,  when  incorporated 
into  the  MVFDS,  provide  the  Air  Force  and  other  users  with  a 
valuable  tool  for  reliable  fire  detection  and  other  applications. 

14.  The  feasibility  of  successfully  completing  a  Phase  II 
effort  has  been  thoroughly  demonstrated. 


82 


SECTION  VII 


RECOMMENDATIONS 


Based  upon  the  conclusions  reached  in  this  study,  the  software 
and  hardware  developments  accomplished,  the  apparent  benefits  of 
the  MVFDS  to  Air  Force  and  other  user  needs,  and  the  demonstration 
and  proof  of  technical  and  applications  feasibility,  it  is 
recommended  that  the  Air  Force  proceed  to  develop  a  MVFDS  product 
through  a  continuing  Phase  II  SBIR  program. 

It  is  also  recommended  that  the  Air  Force  consider  integrating 
the  MVFDS  into  installed  fire  protection  systems,  thus  providing 
for  major  increases  in  system  performance,  reliability,  and 
immunity  to  false  alarms.  In  addition,  the  MVFDS  should  be 
considered  for  new  hangar  developments,  as  well  as  for  other 
applications  such  as  HAS  (USAFE  and  PACAF  shelters) ,  large  body 
cargo  aircraft  (ground  operations) ,  and  other  special  fire 
protection  applications.  The  benefits  offered  by  the  MVFDS  are 
greater  than  those  provided  by  current  detection  systems,  including 
the  involvement  of  man- in-the- loop  TV  monitoring. 

The  types  of  information  generated  by  the  MVFDS  should  be 
considered  in  relation  to  other  subjects  of  security,  assessment, 
monitoring,  and  new  approaches  to  fire  protection  that  involve 
selective  use  of  suppressant  in  only  those  areas  where  the  fire  is 
located. 
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